Using Curl

Curl is a command-line utility and underlying library (libcurl) used to transfer data across a wide range of network protocols, including HTTP, HTTPS, FTP, FTPS, SCP, SFTP, LDAP, MQTT, and more. It supports features like proxy support, user authentication, cookies, file uploads, and SSL connections, making it a go-to tool for developers, system administrators, and anyone working with web APIs or networked services. This guide walks through using curl to perform HTTP requests from your terminal, including downloading files, setting custom headers, working with different HTTP methods, handling redirects, sending JSON payloads, authenticating with servers, and troubleshooting with verbose mode. By the end, you'll be comfortable using curl to test APIs and automate network operations from the command line.

The Short Answer Version

If you just want a quick reference for common curl commands, here's a compact list of essentials:

# Basic GET request
$ curl https://example.com

# Save output to file
$ curl -o page.html https://example.com

# Follow redirects
$ curl -L http://example.com

# Send POST form data
$ curl -d "name=John&age=30" -X POST https://example.com

# Send JSON payload
$ curl -X POST -H "Content-Type: application/json" -d '{"name":"John"}' https://example.com

# Add custom header
$ curl -H "Authorization: Bearer <token>" https://api.example.com

# Basic auth
$ curl -u user:pass https://example.com

Using Curl

This section expands on the quick reference above and explains basic curl operations in more detail, including file downloads, protocol handling, and request formatting.

The basic syntax of the curl command is:

$ curl [options] <URL>

curl Command Options

Detailed Usage

The following sections explain how each curl command works in more detail, with examples and syntax breakdowns.

Download a Webpage

1. Download a webpage and print its source code to the terminal:

$ curl <URL>

Example:

$ curl https://www.example.com/index.html

Use FTP Protocol

1. Retrieve a file using FTP:

$ curl ftp://<host>/<path>

Example:

$ curl ftp://ftp.example.com/public/readme.txt

Save Output to a File

1. Save the downloaded output to a local file instead of printing it to the terminal:

$ curl -o <filename> <URL>
$ curl -O <URL>

• -o: Save output with a custom filename.
• -O: Save output using the remote filename.

Example:

$ curl -o localcopy.html https://example.com/index.html
$ curl -O https://example.com/index.html

Resume an Interrupted Download

1. Resume an interrupted download from where it left off instead of restarting from scratch:

$ curl -C -O <URL>

• -C -: Resume download.
• -O: Save using remote file name.

Example:

$ curl -C -O https://example.com/bigfile.zip

Enable Verbose Output

1. Enable verbose output to debug request/response communication:

$ curl -v <URL>

Example:

$ curl -v https://example.com

Making HTTP Requests

Curl provides a wide range of options to interact with web services, including inspecting headers, following redirects, changing HTTP methods, and customizing request headers.

Get Only Response Headers

1. Fetch only the response headers of a URL:

$ curl -I <URL>

Example:

$ curl -I https://example.com

Follow Redirects

1. Enable following of HTTP 3xx redirects:

$ curl -L <URL>

Example:

$ curl -L http://example.com

Change the HTTP Method

1. Set a custom HTTP method using -X:

$ curl -X [METHOD] <URL>

Example:

$ curl -X DELETE https://example.com/resource/123

Add Custom Headers

1. Use the -H flag to add one or more HTTP headers:

$ curl -H "Header-Name: value" <URL>

Example:

$ curl -H "User-Agent: Mozilla/5.0" -H "Cache-Control: no-cache" https://example.com

Set a Custom User-Agent

1. Use --user-agent to define the request's User-Agent string:

$ curl --user-agent "[Agent String]" <URL>

Example:

$ curl --user-agent "Mozilla/5.0 (Macintosh)" https://example.com

Send Form or JSON Data (POST)

Use curl to send form-encoded or JSON data in HTTP POST requests. This is essential for interacting with REST APIs or submitting data to a server.

1. Send form-encoded or JSON data:

$ curl -d "<key1=value1&key2=value2>" -X POST <URL>
$ curl -X POST -H "Content-Type: application/json" -d '<JSON>' <URL>

Example:

$ curl -d "name=John&age=30" -X POST https://example.com
$ curl -X POST -H "Content-Type: application/json" -d '{"name":"John"}' https://example.com

Basic Authentication

When accessing resources protected by HTTP Basic Auth, use the -u option to include your credentials directly in the request.

1. Send basic auth credentials:

$ curl -u <username>:<password> <URL>

Example:

$ curl -u admin:secret https://example.com

Advanced Usage and Tips

This section covers advanced curl patterns commonly used in scripts, automation, and API integrations.

Exit on HTTP Errors (--fail)

1. Make curl exit silently with a non-zero code when HTTP errors like 404 or 500 occur:

$ curl --fail <URL>

• --fail: Exit silently on HTTP errors (4xx, 5xx).

Example:

$ curl --fail https://example.com/missing-file

Use this in scripts to halt execution when a URL is unreachable or responds with an error.

Measure HTTP Request Time (-w)

1. Print the total time taken to complete a request:

$ curl -w "%{time_total}\n" -o /dev/null -s <URL>

• -w: Custom output after completion.
• %{time_total}: Total transfer time.
• -o /dev/null: Discard body.
• -s: Silent mode to suppress progress.

Example:

$ curl -w "%{time_total}\n" -o /dev/null -s https://example.com

This helps measure API response time or network latency.

Upload Files (-F)

1. Upload files using a multipart form submission:

$ curl -F "key=@filename.ext" <URL>

• -F: Send form field with file.
• @filename.ext: Reads the file from local disk.

Example:

$ curl -F "file=@report.pdf" https://example.com/upload

Use -F for uploading files as part of a form. Use --data-binary if you want to send raw files.

Authenticate with Bearer Token

1. Use bearer tokens to access secured APIs (e.g., OAuth2 access tokens):

$ curl -H "Authorization: Bearer <your_token>" <URL>

Example:

$ curl -H "Authorization: Bearer abc123" https://api.example.com/data

This is required when accessing private endpoints on services like GitHub, Stripe, or cloud APIs.

Quiet or Debug Output (-s, -v)

1. Suppress the progress bar and errors for scripts:

$ curl -s <URL>

2. Show full request and response details for debugging:

$ curl -v <URL>

Example:

$ curl -s https://example.com

$ curl -v https://example.com

Use -s for scripts and logging. Use -v when debugging failures or testing headers.

Next Steps

• Script a health-check that uses curl --fail and -w "%{time_total}" to alert on slow or failing endpoints.
• Combine -H "Authorization: Bearer <token>" with -X POST to scaffold quick API client tests.
• Use -F to exercise file-upload endpoints before writing a full client library.
• Review the official curl documentation for protocol-specific features and scripting patterns.

For the full guide with additional tips, visit the original article on Vultr Docs.