wget It's a tool for downloading files , It's used on the command line . about Linux Users are essential tools , We often have to download some software or restore the backup from the remote server to the local server .

  wget Support HTTP,HTTPS and FTP agreement , have access to HTTP agent . The so-called automatic download means ,wget It can be executed in the background after the user exits the system . This means that you can log in to the system , Start a wget Download task , Then exit the system ,wget Will be performed in the background until the task is completed

   wget Can track HTML The links on the page are downloaded in turn to create a local version of the remote server , Completely rebuild the directory structure of the original site . This is often called ” Recursive download ”.

   wget Very stable , It has strong adaptability in the case of narrow bandwidth and unstable network . If the download fails due to network reasons ,wget Will keep trying , Until the whole file is downloaded . If the server interrupts the download process , It will connect to the server again and continue to download from where it stopped . This is very useful for downloading large files from servers that have limited link times .

wget Common command parameters

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

GNU Wget 1.12, Non interactive network file download tool .

usage : wget [ Options ]... [URL]...

 

Parameters required for long options are also required for short options .

 

Start :

  -V,  --version           Show Wget  Version information of and exit .

  -h,  --help              Print this help .

  -b,  --background        Boot into the background .

  -e,  --execute=COMMAND   Run one ‘.wgetrc’ Command of style .

 

Log in and enter the file :

  -o,  --output-file=FILE    Write the information FILE.

  -a,  --append-output=FILE  Adds information to FILE.

  -d,  --debug               Print a lot of debugging information .

  -q,  --quiet               Quiet mode ( No message output ).

  -v,  --verbose             Detailed output ( This is the default value ).

  -nv, --no-verbose          Close exhaustive output , But not in quiet mode .

  -i,  --input-file=FILE     Download locally or externally FILE Medium URLs.

  -F,  --force-html          Think of the input file as HTML file .

  -B,  --base=URL            Analysis and URL dependent

                             HTML Input file ( from -i -F Option assignment ).

 

download :

  -t,  --tries=NUMBER           Set the number of retries to NUMBER (0 Represent unlimited ).

        --retry-connrefused       Even rejecting a connection is a retry .

  -O,  --output-document=FILE    Write the document to FILE.

  -nc, --no-clobber              Don't download existing files repeatedly .

                                 

  -c,  --continue                 Continue to download some of the downloaded files .

       --progress=TYPE           Select the progress bar type .

  -N,  --timestamping            Gets only new files than local files .

                                  

  -S,  --server-response         Print server response .

       --spider                   Do not download any files .

  -T,  --timeout=SECONDS         Set all timeouts to SECONDS second .

       --dns-timeout=SECS        Set up DNS The search timeout is SECS second .

       --connect-timeout=SECS    Set the connection timeout to SECS second .

       --read-timeout=SECS       Set the read timeout to SECS second .

  -w,  --wait=SECONDS            The waiting interval is SECONDS second .

       --waitretry=SECONDS       Waiting during retries to retrieve files 1..SECONDS second .

       --random-wait             Wait for it to be retrieved 0...2*WAIT second .

       --no-proxy                Close the agent .

  -Q,  --quota=NUMBER            Set the recall quota to NUMBER byte .

       --bind-address=ADDRESS    Bind to... On the local host ADDRESS ( Host name or IP).

       --limit-rate=RATE         Limit the download rate to RATE.

       --no-dns-cache            close DNS Lookup cache .

       --restrict-file-names=OS  Limit the characters in the filename to OS Allowed characters .

       --ignore-case              Match file / Ignore case when Directory .

  -4,  --inet4-only              Connect only to IPv4 Address .

  -6,  --inet6-only              Connect only to IPv6 Address .

       --prefer-family=FAMILY    First connect to the address of the specified protocol

                                 FAMILY by IPv6,IPv4 or none.

       --user=USER               take ftp  and http The user names of are set to USER.

       --password=PASS           take ftp  and http The passwords of are set to PASS.

       --ask-password           Prompt for password .

       --no-iri                close IRI Support .

       --local-encoding=ENC      IRI Use ENC As a local code .

       --remote-encoding=ENC     Use ENC As the default remote encoding .

 

Catalog :

  -nd, --no-directories           Do not create directory .

  -x,  --force-directories        Forced directory creation .

  -nH, --no-host-directories      Don't create a home directory .

       --protocol-directories     Use the protocol name in the directory .

  -P,  --directory-prefix=PREFIX  With PREFIX/... Save the file

       --cut-dirs=NUMBER          Ignore NUMBER A remote directory path .

 

HTTP Options :

       --http-user=USER        Set up http The user is called USER.

       --http-password=PASS    Set up http The password for PASS.

       --no-cache              Don't cache data on the server .

       --default-page=NAME     Change the default page

                               ( The default page is usually “index.html”).

  -E,  --adjust-extension      Save with the appropriate extension HTML/CSS  file .

       --ignore-length         Ignore the head ‘Content-Length’ Area .

       --header=STRING         Insert in the head STRING.

       --max-redirect          Maximum redirection allowed per page .

       --proxy-user=USER       Use USER As agent user name .

       --proxy-password=PASS   Use PASS As a proxy password .

       --referer=URL           stay HTTP The request header contains ‘Referer: URL’.

       --save-headers          take HTTP Save header to file .

  -U,  --user-agent=AGENT      The label is AGENT instead of Wget/VERSION.

       --no-http-keep-alive    Ban HTTP keep-alive( Permanent connection ).

       --no-cookies            Don't use cookies.

       --load-cookies=FILE     Start the conversation from FILE In the load cookies.

       --save-cookies=FILE     Save... After the session cookies to FILE.

       --keep-session-cookies  Load and save session ( Non permanent ) cookies.

       --post-data=STRING      Use POST The way ; hold STRING Send as data .

       --post-file=FILE        Use POST The way ; send out FILE Content .

       --content-disposition   When the local file name is selected

                                allow Content-Disposition Head ( Still in experiment ).

       --auth-no-challenge     send Basic HTTP authentication information

                               without first waiting for the server's

                               challenge.

 

HTTPS (SSL/TLS) Options :

       --secure-protocol=PR     Choose Security Protocol , It can be auto、SSLv2、

                                SSLv3 or TLSv1 One of them .

       --no-check-certificate   Do not verify the server's certificate .

       --certificate=FILE       Client certificate file .

       --certificate-type=TYPE  Client certificate type , PEM or DER.

       --private-key=FILE       Private key file .

       --private-key-type=TYPE  Private key file type , PEM or DER.

       --ca-certificate=FILE    With a group of CA Certified documents .

       --ca-directory=DIR       preservation CA Directory of the authentication Hash list .

       --random-file=FILE       With generation SSL PRNG Files with random data .

       --egd-file=FILE          Used to name... With random data EGD File for socket .

 

FTP Options :

       --ftp-user=USER         Set up ftp  The user is called USER.

       --ftp-password=PASS     Set up ftp  The password for PASS.

       --no-remove-listing     Don't delete ‘.listing’ file .

       --no-glob               be not in FTP Use wildcards in filenames to expand .

       --no-passive-ftp         Ban “passive” transmission mode .

       --retr-symlinks         Recursive directory , Gets the linked file ( Rather than a directory ).

 

Recursive download :

  -r,  --recursive          Specifies a recursive download .

  -l,  --level=NUMBER       Maximum recursion depth ( inf or 0 Represent unlimited , All downloads ).

       --delete-after       Delete the local file when the download is complete .

  -k,  --convert-links      Let the download be available HTML or CSS Links to local files .

  -K,  --backup-converted   Converting files X Before backing it up as X.orig.

  -m,  --mirror             -N -r -l inf --no-remove-listing Abbreviated form .

  -p,  --page-requisites    Download all for display HTML Something like a picture of the page .

       --strict-comments    Turn on HTML The precise treatment of annotations (SGML).

 

Recursive accept / Refuse :

  -A,  --accept=LIST               A comma-separated list of acceptable extension names .

  -R,  --reject=LIST               Comma separated list of extensions to reject .

  -D,  --domains=LIST              Comma separated list of acceptable fields .

       --exclude-domains=LIST      Comma separated list of fields to reject .

       --follow-ftp                 track HTML In document FTP link .

       --follow-tags=LIST          Comma separated tracked HTML List of signs .

       --ignore-tags=LIST          Comma separated ignored HTML List of signs .

  -H,  --span-hosts                Recursion to an external host .

  -L,  --relative                  Only track links that have relationships .

  -I,  --include-directories=LIST  List of allowed directories .

  -X,  --exclude-directories=LIST  List of excluded directories .

  -np, --no-parent                 Do not trace to the parent directory .

Common command display

Use wget -O Download and save under a different filename (-O: Download the file to the corresponding directory , And change the file name )

1

wget -O wordpress.zip http://www.minjieren.com/download.aspx?id=1080

image

Use wget -b Background download

1

wget -b <a href="http://www.minjieren.com/wordpress-3.1-zh_CN.zip">http://www.minjieren.com/wordpress-3.1-zh_CN.zip</a>

1

remarks : You can use the following command to check the download progress :tail -f wget-log

utilize -spider: Simulation download , Will not download , Just check if the website is good

1

[[email protected] ~]# wget --spider  www.baidu.com    # Do not download any files

image

Simulated download print server response

1

[[email protected] ~]# wget -S  www.baidu.com     # Print server response

image

Set the specified number of times

1

2

[[email protected] ~]#  wget -r --tries=2  www.baidu.com       ( Specify the attempt 2 Time ,2 Don't try again )

 [[email protected] ~]# wget -r --tries=2 -q www.baidu.com    ( Specify the attempt , And do not print intermediate results )

image