Introduction
Curl is a versatile and powerful command line tool for transferring data to or from a server using various protocols such as HTTP, HTTPS, FTP, and more. This guide covers the basic usage of curl, including GET, POST, PUT requests, setting HTTP headers, handling files, and using proxies, with detailed descriptions for each command.
TL;DR
You can find a shorter cheat sheet version of this article here.
Table of contents
Open Table of contents
Basic Usage
GET Requests
To perform a simple GET request, which is the most common way to retrieve data from a server, use the following command:
curl http://example.comThis command fetches the content of the specified URL and displays it in the terminal.
For enabling compression, which can reduce the amount of data transferred over the network, use:
curl --compressed http://example.com/This tells curl to request a compressed response (using algorithms like gzip) if the server supports it.
POST Requests
POST requests are used to send data to a server, often for submitting form data or uploading a file. To send simple POST data:
curl -d name=x http://example.comHere, -d name=x sends the data name=x to the server at http://example.com.
For multiple parameters, you can either use multiple -d flags:
curl -d name=x -d surname=y http://example.comor combine them in one string:
curl -d 'name=x&surname=y' http://example.comBoth methods send the parameters name and surname in the POST request.
To send raw data without special interpretation of the @ character:
curl --data-raw "@at" https://example.comThis treats the @ symbol as a regular character rather than indicating a file to upload.
To send URL-encoded data, which ensures that special characters are properly escaped:
curl --data-urlencode name=val http://example.comThis is useful for sending data that includes characters like spaces or symbols.
For sending binary data, such as a file:
curl --data-binary @filename https://example.comThis sends the content of filename as binary data in the POST request.
To send JSON data, commonly used in APIs:
curl --json '{"foo": "bar"}' https://example.com/This sends the JSON object {"foo": "bar"} with the correct Content-Type: application/json header.
For chunked encoding, which is useful for streaming data:
curl -H "Transfer-Encoding: chunked" -d @file http://example.comThis sends the content of file in chunks, suitable for large or dynamically generated data.
To send the output of a command in a POST request:
uname -a | curl -XPOST -d @- http://example.comThis pipes the output of the uname -a command into the POST request.
HEAD Requests
To perform a HEAD request, which retrieves headers only without the body:
curl -I http://example.comThis is useful for checking metadata like the last modified date or content type without downloading the entire resource.
PUT Requests
PUT requests are used to upload data to a server at a specified URL. To upload a file using PUT:
curl -T localfile http://example.com/new/resourceThis uploads localfile to http://example.com/new/resource.
Saving Output to a File
To write the output to a file instead of displaying it in the terminal:
curl -o file.html http://example.comThis saves the content of the URL to file.html.
To save the output using the remote file name:
curl -O http://example.com/file.htmlThis saves the content to a file named file.html, as specified in the URL.
To avoid overwriting existing files:
curl --no-clobber https://example.com/image -o file.pngThis only saves the file if file.png does not already exist.
Additional Options
To skip TLS certificate validation, useful for self-signed certificates:
curl -k https://example.comThis disables certificate verification, making it easier to test secure connections.
For silent output, which suppresses progress and error messages:
curl -s http://example.comThis is useful for scripts where you only need the output content.
To fail fast with no output on server errors:
curl -f http://example.comThis ensures that curl exits immediately on server errors without producing output.
To handle sequences of /../ and /./ in the URL as-is:
curl --path-as-is http://example.com/../../etc/passwdThis prevents curl from normalizing the URL path, useful for testing or accessing specific resources.
Using Proxies
To connect via a proxy server, which can be useful for testing or bypassing network restrictions:
curl -x http://127.0.0.1:8080 http://example.comThis routes the request through the proxy at 127.0.0.1:8080.
For a Socks 4a proxy:
curl --socks4a hostname:1080 https://example.comThis routes the request through a Socks 4a proxy at hostname:1080.
For a Socks5 proxy:
curl --socks5 proxy.example:1080 https://example.comThis routes the request through a Socks5 proxy at proxy.example:1080.
To set a username and password for proxy authentication:
curl --proxy-user username:pass -x http://127.0.0.1:8080 https://example.comThis uses username and pass for authenticating with the proxy.
Setting HTTP Headers
To set a cookie value, which can be necessary for maintaining sessions:
curl -b cookie=value https://example.comThis sends cookie=value with the request.
To set a referer, which indicates the previous page the request came from:
curl -e "https://test.example" https://example.comThis sets the Referer header to https://test.example.
To set a custom User-Agent, useful for mimicking different browsers or clients:
curl -A "New User Agent" https://example.comThis sets the User-Agent header to New User Agent.
or using -H option:
curl -H "User-Agent: Hello" https://example.comTo set a custom header:
curl -H 'X-Custom-Header: HeaderValue' https://example.comThese commands set custom headers, such as User-Agent or X-Custom-Header.
To set the Content-Type header, often required when sending JSON or form data:
curl -d '{}' -H 'Content-Type: application/json' https://example.comThis sets the Content-Type to application/json.
Sending Files
HTTP
To send a file as a form:
curl -F "file=@/path/to/file" http://example.comThis uploads file as a form field.
To send file content as a form field:
curl -F "x=`cat /path/to/file`" http://example.comThis includes the content of the file in the form field x.
To pipe file content directly into curl:
cat /path/to/file | curl -F ":data=@-" http://example.comThis sends the file content via a pipe to curl.
To upload a file:
curl -T /path/to/file http://example.comThis uploads the file to the specified URL.
To send a file in the POST body:
curl -XPOST -d @/path/to/file http://example.comThis sends the file content as the POST request body.
To send binary data in the POST body:
curl --data-binary @binary.data https://example.comThis sends binary data directly.
FTP
To send a file via FTP:
curl -T /path/to/file ftp://server -u user:passwordThis uploads file to the FTP server with the provided username and password.
Conclusion
This guide provides a comprehensive overview of the essential curl commands and options, helping you efficiently transfer data using the curl command line tool. For more detailed information, consult the curl documentation.