command :curl

stay Linux in curl Is a use of URL A file transfer tool that rules work on the command line , It can be said to be a very powerful http Command line tools . It supports file upload and download , It's an integrated transport tool , But by tradition , Customary term url For download tools .

 grammar :# curl [option] [url]

Common parameters :

 Copy code

-A/--user-agent <string>               Set up the user agent to send to the server
-b/--cookie <name=string/file>    cookie String or file read location
-c/--cookie-jar <file>                     After the operation, put cookie Write to this file
-C/--continue-at <offset>             The breakpoint continues
-D/--dump-header <file>               hold header Information is written to the file
-e/--referer                                   Source URL
-f/--fail                                           Do not show... When connection fails http error
-o/--output                                   Write the output to this file
-O/--remote-name                       Write the output to this file , Keep the filename of the remote file
-r/--range <range>                       Search from HTTP/1.1 or FTP Server byte range
-s/--silent                                     Mute mode . Don't export anything
-T/--upload-file <file>                   Upload files
-u/--user <user[:password]>       Set the user and password of the server
-w/--write-out [format]                 After what output is complete
-x/--proxy <host[:port]>               Use... On a given port HTTP agent
-#/--progress-bar                         The progress bar shows the current delivery status 

 Copy code

Example :
1、 Basic usage

# curl

After execution , Of html It will be displayed on the screen
Ps: Due to installation linux Most of the time, there is no desktop installed , It also means no browser , Therefore, this method is often used to test whether a server can reach a website

2、 Save the pages you visit
2.1: Use linux Save the redirection function of

# curl >> linux.html

2.2: have access to curl The built-in option:-o( A lowercase letter ) Save web page

$ curl -o linux.html

After execution, the following interface will be displayed , Show 100% It means the saving is successful

% Total    % Received % Xferd  Average Speed  Time    Time    Time  Current
                                Dload  Upload  Total  Spent    Left  Speed
100 79684    0 79684    0    0  3437k      0 --:--:-- --:--:-- --:--:-- 7781k

2.3: have access to curl The built-in option:-O( Capitalization ) Save the files in the web page
Pay attention to the url Be specific to a document , Or I won't be able to catch it

# curl -O

3、 Test page return value

# curl -o /dev/null -s -w %{http_code}

Ps: In script , It's very common to test whether the website is in normal use

4、 Appoint proxy Server and its port
A lot of time need to use proxy server to get online ( For example, use proxy server to get online or because of using curl Other websites are blocked by others IP At the address ), Fortunately, curl By using the built-in option:-x To support setting up agents

# curl -x

Some websites use cookie To record session Information . about chrome A browser like this , It's easy to deal with cookie Information , But in curl As long as the relevant parameters are added, it can be easily handled cookie
5.1: preservation http Of response Inside cookie Information . built-in option:-c( A lowercase letter )

# curl -c cookiec.txt

After execution cookie The information is stored in cookiec.txt Inside the

5.2: preservation http Of response Inside header Information . built-in option: -D

# curl -D cookied.txt

After execution cookie The information is stored in cookied.txt Inside the

Be careful :-c( A lowercase letter ) Produced cookie and -D Inside cookie It's different .

5.3: Use cookie
Many websites monitor you through cookie Information to determine whether you visit their website according to the rules , So we need to use preserved cookie Information . built-in option: -b

# curl -b cookiec.txt

6、 Mimic browser
Some websites need to use specific browsers to visit them , Some also need to use certain versions .curl built-in option:-A Let's specify a browser to visit the website

# curl -A "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.0)"

In this way, the server side will think that it is using IE8.0 To visit

7、 forge referer( Stealing chain )
Many servers will check http Access to the referer To control access . such as : You are the first to visit the home page , Then visit the email page on the home page , I'm here to visit the mailbox referer Address is the page address after the success of visiting the home page , If the server finds that it has access to the mailbox page referer The address is not the address of the home page , It's a rogue company
curl Built in option:-e Let's set referer

# curl -e ""

This will make the server think you are from Click on a link to

8、 Download the file
8.1: utilize curl Download the file .
# Built in option:-o( A lowercase letter )

# curl -o dodo1.jpg

# Built in option:-O( Capitalization )

# curl -O

This saves the file locally under the name of the server

8.2: Cycle download
Sometimes the download image can be part of the previous name is the same , The name of the last vertebra is different

# curl -O[1-5].JPG

This will put dodo1,dodo2,dodo3,dodo4,dodo5 Save it all

8.3: Download rename

# curl -O{hello,bb}/dodo[1-5].JPG

Because of the download hello And bb The file names in are dodo1,dodo2,dodo3,dodo4,dodo5. So the second download will overwrite the first one , So you need to rename the file .

# curl -o #1_#2.JPG{hello,bb}/dodo[1-5].JPG

In this way hello/dodo1.JPG When you download the file, it will become hello_dodo1.JPG, Other documents and so on , Thus effectively avoid the file being overwritten

8.4: Download in blocks
Sometimes the download is bigger , We can download it in sections at this time . Built in option:-r

# curl -r 0-100 -o dodo1_part1.JPG
# curl -r 100-200 -o dodo1_part2.JPG
# curl -r 200- -o dodo1_part3.JPG
# cat dodo1_part* > dodo1.JPG

So you can see dodo1.JPG Content.

8.5: adopt ftp Download the file
curl Can pass ftp Download the file ,curl There are two ways to get from ftp Download the syntax in

# curl -O -u  user name : password
# curl -O ftp:// user name : password

8.6: Show download progress bar

# curl -# -O

8.7: Download progress information will not be displayed

# curl -s -O

9、 Breakpoint continuation
stay windows in , We can use software like Xunlei for breakpoint retransmission .curl Can be built through option:-C The same effect can be achieved
If you are downloading dodo1.JPG In the process, the line suddenly dropped , You can use the following methods to renew

# curl -C -O

10、 Upload files
curl It's not just about downloading files , You can also upload files . Through the built-in option:-T To achieve

# curl -T dodo1.JPG -u  user name : password

In this way, I would like to ftp The server uploaded the file dodo1.JPG

11、 Show grab error

# curl -f

The other parameters ( Reprint ):

 Copy code

-a/--append                         When uploading files , Attach to target file
--anyauth                             have access to “ whatever ” Authentication methods
--basic                                 Use HTTP Basic verification
-B/--use-ascii                       Use ASCII Text transfer
-d/--data <data>                  HTTP POST Way to transmit data
--data-ascii <data>             With ascii The way post data
--data-binary <data>           In a binary way post data
--negotiate                           Use HTTP Authentication
--digest                         Use digital authentication
--disable-eprt                   No use EPRT or LPRT
--disable-epsv                   No use EPSV
--egd-file <file>               For random data (SSL) Set up EGD socket route
--tcp-nodelay                   Use TCP_NODELAY Options
-E/--cert <cert[:passwd]>       Client certificate file and password  (SSL)
--cert-type <type>               Certificate file type  (DER/PEM/ENG) (SSL)
--key <key>                     Private key filename  (SSL)
--key-type <type>               Private key file type  (DER/PEM/ENG) (SSL)
--pass  <pass>                   Private key password  (SSL)
--engine <eng>                   The encryption engine uses  (SSL). "--engine list" for list
--cacert <file>                CA certificate  (SSL)
--capath <directory>            CA Objective    (made using c_rehash) to verify peer against (SSL)
--ciphers <list>                SSL password
--compressed                     Asking for a return is a compressed situation  (using deflate or gzip)
--connect-timeout <seconds>     Set the maximum request time
--create-dirs                   Create a directory hierarchy of local directories
--crlf                           Upload is to LF Into a CRLF
--ftp-create-dirs               If the remote directory does not exist , Create a remote directory
--ftp-method [multicwd/nocwd/singlecwd]     control CWD Use
--ftp-pasv                       Use  PASV/EPSV  Instead of ports
--ftp-skip-pasv-ip               Use PASV When , Ignore this IP Address
--ftp-ssl                       Try to use  SSL/TLS  To carry out ftp The data transfer
--ftp-ssl-reqd                   Asked to use  SSL/TLS  To carry out ftp The data transfer
-F/--form <name=content>         simulation http Form submission data
-form-string <name=string>       simulation http Form submission data
-g/--globoff                     Disable URL sequence and range usage {} and []
-G/--get                         With get To send data
-h/--help                       help
-H/--header <line>               Custom header information is passed to the server
--ignore-content-length         Neglected HTTP Length of header information
-i/--include                     Output includes protocol Header information
-I/--head                       Show only document information
-j/--junk-session-cookies       Ignore... When reading files session cookie
--interface <interface>         Use the specified network interface / Address
--krb4 <level>                   Use... With the specified security level krb4
-k/--insecure                   Allow certificates not to be used to SSL Site
-K/--config                     Specified profile read
-l/--list-only                   List ftp The name of the file in the directory
--limit-rate <rate>             Set the transmission speed
--local-port<NUM>               Force local port number
-m/--max-time <seconds>         Set the maximum transfer time
--max-redirs <num>               Set the maximum number of directories to read
--max-filesize <bytes>           Set the maximum number of files to download
-M/--manual                     Display full manual
-n/--netrc                       from netrc Read the user name and password in the file
--netrc-optional                 Use  .netrc  perhaps  URL To cover -n
--ntlm                           Use  HTTP NTLM  Authentication
-N/--no-buffer                   Disable buffered output
-p/--proxytunnel                 Use HTTP agent
--proxy-anyauth                 Choose any proxy authentication method
--proxy-basic                   Use basic authentication on agents
--proxy-digest                   Use digital authentication on agents
--proxy-ntlm                     Use... On agents ntlm Authentication
-P/--ftp-port <address>         Use port address , Instead of using PASV
-Q/--quote <cmd>                 Before file transfer , Send command to server
--range-file                     Read (SSL) A random file of
-R/--remote-time                 When generating files locally , Keep remote files for
--retry <num>                   When there is a problem with the transmission , The number of retries
--retry-delay <seconds>         When there is a problem with the transmission , Set retry interval
--retry-max-time <seconds>       When there is a problem with the transmission , Set the maximum retry time
-S/--show-error                 Display error
--socks4 <host[:port]>           use socks4 Agent gives host and port
--socks5 <host[:port]>           use socks5 Agent gives host and port
-t/--telnet-option <OPT=val>    Telnet Option is set
--trace <file>                   On the specified file debug
--trace-ascii <file>            Like -- Tracking but not hex Output
--trace-time                     track / Detailed output , Add time stamp
--url <URL>                    Spet URL to work with
-U/--proxy-user <user[:password]>   Set the proxy user name and password
-V/--version                     Display version information
-X/--request <command>           What order to give
-y/--speed-time                 The time it takes to give up the speed limit . The default is 30
-Y/--speed-limit                 Stop transmission speed limit , Speed time ' second
-z/--time-cond                   Transmission time settings
-0/--http1.0                     Use HTTP 1.0
-1/--tlsv1                       Use TLSv1(SSL)
-2/--sslv2                       Use SSLv2 Of (SSL)
-3/--sslv3                       The use of SSLv3(SSL)
--3p-quote                      like -Q for the source URL for 3rd party transfer
--3p-url                         Use url, Make a third-party transmission
--3p-user                       Use user name and password , Make a third-party transmission
-4/--ipv4                       Use IP4
-6/--ipv6                       Use IP6

 Copy code

from :