Get page content
> curl https://json.im
Show HTTP headers
> curl -I https://json.im HTTP/1.1 200 OK Server: openresty Date: Fri, 04 Jun 2021 07:38:32 GMT Content-Type: text/html Content-Length: 12864 Last-Modified: Thu, 29 Apr 2021 01:39:01 GMT Connection: keep-alive Vary: Accept-Encoding ETag: "608a0e35-3240" Expires: Fri, 04 Jun 2021 08:38:32 GMT Cache-Control: max-age=3600 Accept-Ranges: bytes
The HTTP header and file content are displayed at the same time. Use the - i option
> curl -i https://json.im
Save link to file
We can use the > symbol to redirect the output to a local file.
> curl https://json.im > index.html
You can also save the contents to a file through curl's - o/-O option.
- -O (lowercase o): the result will be saved to the file name provided in the command line
- -O (uppercase o): the file name in the URL will be used as the file name to save the output
> curl -o index.html https://json.im > curl -O https://json.im/index.html
Note: when using the - O option, you must ensure that the file name is included at the end of the link, otherwise curl cannot save the file correctly. If there is no file name in the link, you should specify the file name manually with the - O option or use the redirection symbol
Download multiple files at the same time
We can use the - O or - O option to specify multiple links at the same time, and write the command in the following format:
> curl -O html https://json.im/1.html -O html https://json.im/2.html > curl -o p1.html https://json.im/1.html -o p2.html https://json.im/2.html
Follow link redirection with - L
If we directly use curl to open some links that are reset backward, we can't get the web content we want in this case. For example:
> curl http://json.im 301 Moved Permanently 301 Moved Permanently openresty
When we open the link through the browser, we will automatically jump to https://json.im . At this point, what we want curl to do is follow the jump of the link like a browser to get the final web content. We can add the - L option to the command to follow the link redirection:
> curl -L http://json.im
Use - A to customize the user agent
We can use - A to customize the user * * *. For example, the following command will disguise as Android Firefox browser to request web pages:
> curl -A "Mozilla/5.0 (Android; Mobile; rv:35.0) Gecko/35.0 Firefox/35.0" https://json.im
Use - H to customize the header
When we need to pass a specific header, we can follow the following command:
> curl -H "Referer: https://rumenz.com" -H "User-Agent: Custom-User-Agent" https://json.im
Pass Cookie in header
> curl -H "Cookie: JSESSIONID=xxx" https://json.im
Use - c to save cookies
When using cURL to access a page, cookies will not be saved by default
> curl -c "cookie.txt" https://json.im
Use - b to read cookies
> curl -b "cookie.txt" https://json.im
Send POST request with - d
There is a login page https://json.im/login , you only need to submit your user name and password to log in. We can use cURL to complete this POST request, - d to specify the data to be sent, and - X to specify the way to send the data
> curl -d "userName=rumenz&passwd=123456" -X POST https://json.im/login
When - d is used, if - X is omitted, the default mode is POST:
> curl -d "userName=rumenz&passwd=123456" https://json.im/login
Read the data.txt text from the file
> curl -d "@data.txt" https://json.im/upload
Resume interrupted Download
> curl -C - -O https://json.im/jdk.tar.gz
Download URL from file
If you use curl with xargs, you can download the file from the list of URL s in the file.
> xargs -n 1 curl -O < urls.txt
CURL set proxy
> curl https://json.im/ -U user:password -x 127.0.0.1:3128
If your proxy does not require authentication, you can skip - U user:password.
For example, Nginx configures the agent
server { listen 3128; resolver 8.8.8.8; proxy_connect; proxy_connect_allow 443 563; proxy_connect_connect_timeout 10s; proxy_connect_read_timeout 10s; proxy_connect_send_timeout 10s; location / { proxy_pass http://$host; proxy_set_header Host $host; } }
Upload files to ftp
> curl -u username:password -T jdk.tar.gz ftp://json.im
Modify name resolution
> curl --resolve json.im:443:127.0.0.1 https://json.im:443/
yes https://json.im Your query will tell curl to request the site from 127.0.0.1 instead of using DNS or the / etc/hosts file.
Limit download rate
> curl --limit-rate 100K https://json.im/jdk.tar.gz -O
HTTP authentication
Some domains require HTTP authentication, and curl requires the – user parameter.
> curl --user name:passwd https://json.im
Original link: https://rumenz.com/rumenbiji/linux-curl-skills.html
WeChat official account: entry station