Skip to content

Wget cheatsheet

Basic usage

Download the URL

wget https://example.com

Concatenate the output and write to a single file

wget -O file http://example.com

Ignore certificate errors

wget --no-check-certificate http://example.com

Ignore robots.txt file

wget -e robots=off http://example.com

Set maximum number of redirects

wget --max-redirect=number http://example.com

Cookies

Disable cookies

wget --no-cookies http://example.com

Set custom cookie value

wget --no-cookies --header "Cookie: name=value"

Save cookies to file

wget --save-cookies cookies.txt https://example.com

Load cookies from file

wget --load-cookies cookies.txt https://example.com

Headers

Send additional HTTP header (can be specified multiple times)

wget --header='Custom-header: test' https://example.com

Set referer

wget --referer=url http://example.com

User agent

wget -U user-agent-string http://example.com

Disable cache (send Cache-Control and Pragma headers with no-cache value)

wget --no-cache https://example.com

Save headers at the beginning of the file

wget --save-headers http://example.com

Timeouts

Define timeout of the request

wget -T 10 http://example.com

The -T combines all the options below:

  • --dns-timeout - DNS loookup
  • --connect-timeout - TCP connection
  • --read-timeout - idle time

Download files

Download all PDF files, ignore robots.txt file

wget -e robots=off -r -l1 -A.pdf -nd http://server

Mirror the website

wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.com

or shorter version

wget -m -k -E -p -np http://example.com