How to Make Your Server Serve Compressed Responses: A Step-by-Step Guide to Saving Bandwidth

One header in an HTTP request can slash the amount of data you download by several times. We're talking about negotiating compression between client and server. If you work with proxies and pay for every gigabyte of traffic, this topic directly hits your wallet. In this guide, we'll break down how to get your server to serve compressed responses, how to measure the real savings for your specific task, and what pitfalls to watch out for.

Introduction: One Header Can Slash Your Bandwidth

HTTP compression works surprisingly simply. The client tells the server which compression algorithms it understands. The server picks a suitable one, compresses the response body, and sends it. The client decompresses the data on its end. The result: what travels over the network isn't the original HTML or JSON, but its compressed version, which often weighs three to five times less.

What you'll get by the end. After reading this guide, you'll be able to enable compression on the client side, verify that the server is actually serving a compressed response, measure the bandwidth savings in bytes before and after, and diagnose situations where compression isn't working for some reason. All of this is critical when you're routing traffic through a Proxeon proxy and paying per gigabyte.

Who this guide is for. This material is aimed at developers, scraping specialists, automation engineers, and anyone writing HTTP clients that work through proxies. It's intermediate level. We won't break down everything that contributes to overall bandwidth usage—that deserves its own article. Here, we're strictly focused on compression.

What you need to know beforehand. A basic understanding of what an HTTP request and response are, what headers are, and how to run a command in a terminal. If you've ever made a request with curl or written a script in Python or Node.js, you'll be fine.

How long it'll take. Reading and working through all the examples will take about sixty minutes. You'll see your first practical results within ten minutes when you compare the size of a compressed vs. uncompressed response with your own eyes.

Preparation

Before you start, make sure you have everything you need. This will save time and prevent errors halfway through.

Required Tools

• curl version 7.72 or newer. In 2026 builds, brotli and zstd are supported out of the box on most systems.
• Python version 3.10 or newer with the requests library installed, plus the brotli or brotlicffi package for brotli support.
• Node.js version 18 or newer. The built-in zlib module handles gzip, deflate, and brotli.
• Access to the Proxeon proxy if you want to measure savings in the exact conditions you work in every day.
• Any text editor for writing scripts.

System Requirements

Any modern system works: Windows, macOS, or Linux. All examples are cross-platform. Two gigabytes of RAM is plenty. There are no special CPU requirements, though compressing large volumes with brotli at maximum level can noticeably load a single core.

What to Install and Verify

1. Open a terminal and run the curl version check command:

   curl --version

2. Look at the first line of output. That shows the version. Make sure the list of features includes brotli and, if possible, zstd.
3. Check Python with

   python --version

   and install dependencies:

   pip install requests brotli

4. Check Node.js with

   node --version

Tip: If curl on your system isn't built with brotli, install it via your OS's official package manager. In 2026 enterprise builds, brotli is almost always enabled by default.

Backups. In this guide, we're not changing production server configuration or touching live data. We're only sending requests and measuring responses. So backups aren't needed. However, if you later decide to enable compression on your own web server, make sure to save the original config file before editing.

✅ Check: All three tools respond to the version check command and show a version number. Preparation is complete—let's move on.

Basic Concepts in Plain English

Before we start pressing buttons, let's get the terminology straight. This will take five minutes but will save you from confusion later.

How Compression Negotiation Works

HTTP compression relies on three headers. Understanding their roles solves most problems.

Accept-Encoding from the client. This is the header your client sends with the request. It lists the compression algorithms the client can decompress. For example, the string

Accept-Encoding: gzip, br, zstd

Content-Encoding in the response. This is the header the server adds to its response. It tells you exactly which algorithm was used to compress the body. If you see

Content-Encoding: br

Vary on the server side. The Vary header tells caches and proxies that the response depends on certain request headers. For compression, the critical value is

Vary: Accept-Encoding

The Key Idea of Negotiation

Remember this sequence. The client asks via Accept-Encoding. The server decides and responds via Content-Encoding. Intermediate nodes rely on Vary. If even one link in this chain is broken, you'll get an uncompressed response and overpay for bandwidth.

Tip: Even if your HTTP library adds Accept-Encoding automatically, always check the actual response header. Automation sometimes stays silent while bandwidth flies out the window.

gzip, deflate, brotli, zstd: Algorithm Comparison

There are four common compression algorithms for HTTP. They differ in compression ratio and CPU load. Let's break down each one and put the numbers in a table.

gzip

The oldest and most universal. Supported absolutely everywhere: any server, any client, any proxy. Provides a good compression ratio on text data with moderate CPU load. If you're unsure what to choose, start with gzip.

deflate

A close relative of gzip, using the same algorithm internally but with a different wrapper. In practice, it's rarer and sometimes implemented with bugs on servers. Compression ratio is roughly the same as gzip. There's no particular reason to choose deflate over gzip.

brotli

A modern algorithm designed specifically for the web. On text data, it compresses noticeably better than gzip, especially on HTML, CSS, and JSON. The trade-off is higher CPU load when compressing at maximum levels. Decompression is fast. It's denoted as br in headers.

zstd

A fast modern algorithm with flexible level tuning. It offers a compression ratio on par with brotli but works faster in terms of compression speed. Web support is growing—by 2026, most current servers and clients understand it. It's denoted as zstd.

Comparison Table on a Typical Text Response

Let's take a hypothetical JSON response at 1000 kilobytes uncompressed. The numbers are approximate—they depend on the content—but the order of magnitude is accurate.

• No compression: 1000 KB, 0% savings, minimal CPU load.
• gzip level 6: about 190 KB, roughly 81% savings, low CPU load.
• deflate level 6: about 195 KB, roughly 80% savings, low CPU load.
• brotli level 5: about 165 KB, roughly 83% savings, medium CPU load.
• brotli level 11: about 140 KB, roughly 86% savings, high CPU load.
• zstd level 3: about 175 KB, roughly 82% savings, low CPU load.
• zstd level 19: about 150 KB, roughly 85% savings, medium CPU load.

The conclusion is simple. On text data, any compression reduces bandwidth roughly fivefold. The difference between algorithms is within a few percentage points. So, for saving bandwidth through a proxy, the key isn't which algorithm you use—it's the fact that compression is enabled at all.

Tip: If you're only downloading data, not serving it, the CPU load on your side is only for decompression, which is fast for all algorithms. That means you can safely request the most compression-efficient option.

✅ Check: You understand that brotli and zstd are usually slightly more efficient than gzip on text, but gzip is the most compatible. That's enough for practice.

Step 1: Send a Request and See If the Response Is Compressed

Goal of this step. Learn to see the Content-Encoding headers and the actual body size. Without this, you can't measure savings.

Check with curl

1. Open a terminal.
2. First, request the resource without compression to find out the original size. Run:

   curl -s -o /dev/null -w "%{size_download}" https://example.com/api/data

3. Write down the number. This is the size of the uncompressed response in bytes.
4. Now ask for compression. The --compressed flag automatically adds Accept-Encoding and decompresses the response:

   curl -s --compressed -o /dev/null -w "%{size_download}" https://example.com/api/data

5. Compare the two numbers. The second should be noticeably smaller for the same content.

Important: The size_download metric in curl with --compressed reflects the size of already-decompressed data, not what actually traveled over the network. To see the real network volume, use the response headers and intermediate measurements, which we'll cover in the measurement step.

Look at Response Headers

1. Run a request with the header output flag:

   curl -s -I --compressed https://example.com/api/data

2. Find the Content-Encoding line in the output. If it says gzip, br, or zstd, the server served a compressed response.
3. Find the Vary line. The presence of Accept-Encoding in it means caching is configured correctly.

Tip: To explicitly request a specific algorithm, add the header manually:

curl -s -H "Accept-Encoding: br" -I https://example.com/api/data

Working Through the Proxeon Proxy

To measure savings in real-world conditions, route your request through the proxy. In curl, that's done with the -x flag:

curl -s --compressed -x http://username:password@proxeon_address:port -o /dev/null -w "%{size_download}" https://example.com/api/data

This way, you'll see if compression survives the proxy and whether anything strips headers along the way.

Expected result. You see two numbers: size without compression and size with compression. And you see the Content-Encoding header in the response. That means the mechanism works.

Possible issue: If both numbers are the same and Content-Encoding is missing, the server didn't serve compression. We'll cover the reasons in a separate section.

✅ Check: The response with --compressed includes a Content-Encoding line with one of the algorithms. Step complete.

Step 2: Measure Real Savings with Python

Goal of this step. Get precise numbers for network traffic before and after compression with a working script you can apply to your own task.

Simple Measurement with requests

The requests library adds Accept-Encoding and decompresses the response by default. To measure the actual network size, we'll look at the length of the raw content before decompression.

1. Create a file measure.py.
2. Paste in the following code:

import requests
url = "https://example.com/api/data"
# Request without compression
r_plain = requests.get(url, headers={"Accept-Encoding": "identity"})
size_plain = len(r_plain.content)
# Request with compression
r_comp = requests.get(url, headers={"Accept-Encoding": "gzip, br, zstd"})
encoding = r_comp.headers.get("Content-Encoding", "none")
size_wire = int(r_comp.headers.get("Content-Length", 0))
size_body = len(r_comp.content)
print("Without compression, bytes:", size_plain)
print("Compression algorithm:", encoding)
print("Approximate wire bytes:", size_wire)
print("After decompression, bytes:", size_body)
if size_plain and size_wire:
saved = 100 - size_wire * 100 
print("Bandwidth savings, percent:", saved)

Note: The Content-Length value isn't always present, especially with chunked transfer. If it's missing, use the more precise method below.

Precise Network Volume Measurement

To measure exactly what traveled over the network, disable automatic decompression and count raw stream bytes.

import requests
url = "https://example.com/api/data"
sess = requests.Session()
# Disable auto-decompression to see raw size
r = sess.get(url, headers={"Accept-Encoding": "gzip, br, zstd"}, stream=True)
raw_bytes = 0
for chunk in r.raw.stream(8192, decode_content=False):
raw_bytes += len(chunk)
print("Algorithm:", r.headers.get("Content-Encoding", "none"))
print("Raw bytes over the wire:", raw_bytes)

Compare raw_bytes with the size of the uncompressed request. The difference is your real savings.

Measurement Through the Proxeon Proxy

Add the proxies parameter to get numbers in production-like conditions:

proxies = {
"http": "http://username:password@proxeon_address:port",
"https": "http://username:password@proxeon_address:port",
}
r = sess.get(url, headers={"Accept-Encoding": "gzip, br, zstd"}, proxies=proxies, stream=True)

Tip: Run the measurement several times and average the results. Dynamic responses vary in size, so a single measurement can be misleading.

Expected result. The script prints the compression algorithm and the number of raw bytes, which is smaller than the uncompressed response size. If savings are seventy percent or more on a text resource, everything is working as it should.

✅ Check: In the output, the algorithm isn't the word "none," and raw bytes are fewer than without compression. Great.

Step 3: The Same Measurement with Node.js

Goal of this step. Get a working measurement tool in JavaScript for those writing clients in Node.

Measuring the Raw Stream

1. Create a file measure.js.
2. Paste in code that counts bytes before decompression:

const https = require('https');
const url = 'https://example.com/api/data';
function measure(acceptEncoding) {
 return new Promise((resolve) => {
const req = https.get(url, { headers: { 'Accept-Encoding': acceptEncoding } }, (res) => {
 let bytes = 0;
 res.on('data', (chunk) => { bytes += chunk.length; });
 res.on('end', () => {
resolve({ enc: res.headers['content-encoding'] || 'none', bytes });
 });
});
req.end();
 });
}
(async () => {
 const plain = await measure('identity');
 const comp = await measure('gzip, br, zstd');
 console.log('Without compression, bytes:', plain.bytes);
 console.log('Algorithm:', comp.enc);
 console.log('Compressed over the wire, bytes:', comp.bytes);
 const saved = Math.round(100 - (comp.bytes * 100) / plain.bytes);
 console.log('Savings, percent:', saved);
})();

Important: In this example, we read the raw stream from res and don't connect zlib. That means the bytes counter shows exactly the network volume. This is precisely what you pay your proxy provider for.

Manual Decompression in Node

If you also need to read the content, decompress the stream with the built-in zlib module:

const zlib = require('zlib');
let stream = res;
const enc = res.headers['content-encoding'];
if (enc === 'gzip') stream = res.pipe(zlib.createGunzip());
else if (enc === 'br') stream = res.pipe(zlib.createBrotliDecompress());
else if (enc === 'deflate') stream = res.pipe(zlib.createInflate());

Tip: For zstd, newer Node.js versions include zlib.createZstdDecompress. If your version doesn't have it, update Node or use a separate npm package.

Expected result. Node prints savings as a percentage, comparable to what you got with Python and curl.

✅ Check: Three tools give similar savings figures. Your measurements are reliable.

Step 4: Why the Response Comes Back Uncompressed

Goal of this step. Learn to diagnose why compression didn't kick in and fix the root cause. This is the most common practical problem.

Case One: The Client Didn't Ask

The most common reason. Your HTTP client didn't send the Accept-Encoding header, so the server honestly served an uncompressed response. This happens when you manually build headers and forget about compression, or when you use a low-level socket.

1. Check what's actually going to the server. In curl, add the -v flag and look for the Accept-Encoding line among outgoing headers.
2. If the line isn't there, add it explicitly with -H or the --compressed flag.
3. In Python, make sure you haven't overridden the header with an empty value.

Note: Some libraries stop adding their default headers when you manually set any header. If you're setting User-Agent by hand, check that Accept-Encoding hasn't disappeared along with it.

Case Two: The Server Doesn't Support It

The server may not support compression at all, or it may not support the specific algorithm you requested. In that case, it serves an uncompressed response, which is allowed by the standard.

1. Try different algorithms one at a time: first gzip, then br, then zstd.
2. See which one produces a Content-Encoding header in the response.
3. If none do, the server doesn't serve compression. You can't change someone else's server, but you might be able to choose a different endpoint or API if one exists.

Tip: Many APIs only serve compression for responses above a certain size. Small responses are often left uncompressed on purpose because the overhead of compression outweighs the benefit.

Case Three: An Intermediary Stripped the Header

Between you and the server, there might be a caching node, gateway, or proxy that removes or modifies compression headers. Sometimes an intermediary decompresses the response for its own needs and serves you the already-decompressed version.

1. Make a direct request and a proxied request, then compare the Content-Encoding headers.
2. If compression exists directly but not through the intermediary, the intermediary is interfering.
3. When working through the Proxeon proxy, compression is passed through transparently, so if you see a difference, look for the problem on the target server's side or its CDN.

Expected result. You know exactly which of the three links is losing compression, and you understand what to do about it.

✅ Check: You can explain in one sentence why a specific response came back uncompressed. Diagnostics mastered.

Step 5: Pitfalls When Working with Compression

Goal of this step. Avoid subtle errors that break data or skew measurements.

Double Compression

Sometimes content is already compressed at the application level, and the server compresses it again. Or vice versa: you compress the request body, and the proxy or library does it again. Double compression gives almost no benefit, sometimes even increases size, and definitely wastes CPU cycles.

1. Check if the response has two values in Content-Encoding, like gzip inside br.
2. Don't manually compress what the library will already compress automatically.
3. If you see a chain of encodings, decompress them in reverse order.

Corrupted Stream

A dropped connection or an intermediary error can result in an incomplete compressed stream. When decompressing, you'll get an error or truncated data.

1. Always wrap decompression in error handling.
2. On decompression error, retry the request instead of trying to read corrupted data.
3. Compare the actual size against Content-Length if it's present.

Note: Never save partially decompressed data as a result. Corrupted JSON looks plausible but will cause hidden errors further down the pipeline.

Manual Decompression When the Library Doesn't Do It

If you disabled auto-decompression for accurate measurement, or you're using a low-level client, you'll need to decompress manually. Example in Python:

import gzip, brotli, zlib
def decompress(data, encoding):
if encoding == "gzip":
return gzip.decompress(data)
if encoding == "br":
return brotli.decompress(data)
if encoding == "deflate":
return zlib.decompress(data)
return data

Tip: If deflate decompression fails with an error, try calling zlib.decompress with a wbits parameter of minus fifteen. Some servers serve raw deflate without the zlib wrapper.

Expected result. You can manually decompress any supported format and handle stream errors correctly.

✅ Check: Your script doesn't crash on a corrupted response; it retries the request. Resilience achieved.

Step 6: What's Pointless to Compress

Goal of this step. Don't waste CPU and time compressing things that are already compressed. This saves resources without losing any bandwidth benefit.

Already-Compressed Formats

Some data is stored in formats that already use compression internally. Trying to compress them again is pointless: the gain is a fraction of a percent, and you'll load your CPU for nothing.

• Images: JPEG, PNG, WebP, AVIF are already compressed within their format.
• Video: MP4, WebM, MKV contain heavily compressed video streams.
• Audio: MP3, AAC, OGG are compressed by nature.
• Archives: ZIP, RAR, 7z, gz are already compressed containers.

For these resources, Accept-Encoding won't save you anything on the body. Moreover, servers typically don't compress them again either, because they're configured by MIME-type lists.

What's Worth Compressing

• HTML, CSS, JavaScript.
• JSON and XML API responses.
• Plain text, CSV, logs.
• SVG, because it's essentially text.

Tip: If you're mostly downloading media files, compression savings will be nearly zero. What helps there is something else: requesting only the sizes and formats you need, not the fact of compression itself. But that's a topic for bandwidth management, not compression per se.

Expected result. You don't waste resources compressing binary formats and only apply compression where it truly helps.

✅ Check: You can tell in a second whether compressing a specific response type makes sense. Understanding achieved.

Verifying the Result

Time to make sure everything is set up correctly. Go through the checklist.

Functionality Checklist

1. Your client sends the Accept-Encoding header with the algorithms you need.
2. The response includes Content-Encoding with one of those algorithms on text resources.
3. The measurement script shows network volume at least seventy percent less than uncompressed for text.
4. Through the Proxeon proxy, compression is preserved, and numbers match the direct request.
5. Decompression doesn't error out, and data reads correctly.
6. You're not trying to compress binary formats.

How to Test

1. Take three different endpoints: one with JSON, one with HTML, one with an image.
2. Run each through your measurement script.
3. Verify that savings are high on the first two and near zero on the third.

Success indicators. For a text API, you consistently see savings in the seventy to ninety percent range. For media, savings are minimal, and that's normal. Through the proxy, numbers don't degrade.

✅ Check: All six checklist items are met. Compression is configured and measured correctly.

Common Errors and Solutions

Here are frequent problems in a problem-cause-solution format.

The Response Isn't Compressed Even Though Accept-Encoding Is Sent

Cause: The server doesn't support compression for this content type or for responses of this size. Solution: Try a different algorithm and make sure the resource is text and large enough.

Savings Aren't Visible in curl; size_download Doesn't Change

Cause: The --compressed flag shows the size after decompression. Solution: Measure network volume with a separate script without auto-decompression, as in the Python and Node steps.

The Library Returns Garbage Instead of Text

Cause: You disabled auto-decompression but didn't decompress the stream manually. Solution: Determine Content-Encoding and apply the appropriate decompression function.

Error Decompressing deflate

Cause: The server served raw deflate without the wrapper. Solution: Call decompression with a wbits parameter of minus fifteen.

Compression Disappears Through the Proxy

Cause: A node on the path is modifying headers, most often the target site's CDN. Solution: Compare direct vs. proxied requests and identify the link. The Proxeon proxy passes headers transparently, so the problem is usually server-side.

Response Size Varies Between Repeated Measurements

Cause: Content is dynamic and changes from request to request. Solution: Average multiple measurements and compare on identical request parameters.

Content-Length Is Missing

Cause: Chunked streaming doesn't specify length in advance. Solution: Count bytes manually as you read the stream, as in the script examples.

Double Compression Wastes CPU

Cause: Content is compressed twice at different levels. Solution: Remove manual compression where the library or server does it.

Additional Possibilities

Once you've mastered the basics, there's room to grow.

Choosing Algorithm Priorities

In the Accept-Encoding header, you can specify weights with q-values to hint at preferences. For example,

Accept-Encoding: zstd;q=1.0, br;q=0.9, gzip;q=0.8

Caching Decompressed Data

If you repeatedly hit the same resource, cache the decompressed result on your side. This reduces both bandwidth and the CPU load from repeated decompression.

Mass Measurement Across a URL List

Wrap the measurement script in a loop over a list of endpoints and compile a savings table. This helps you find the heaviest uncompressed responses in your project and tackle them first.

Tip: Keep a daily log of bytes saved. Multiply by your per-gigabyte price to see the tangible financial benefit of compression when working through a proxy.

✅ Check: You can compile a summary savings table across multiple resources with a single script run.

FAQ

Should I always request brotli instead of gzip?

If you're only downloading, decompression is fast for all algorithms, so you can request the most efficient one. In practice, put all three in Accept-Encoding: gzip, br, zstd. The server will pick the best available.

How much bandwidth does compression save on a text API?

Typically seventy to ninety percent. That means a response that weighed a gigabyte will travel as 150 or 200 megabytes. The savings when paying per gigabyte are direct and noticeable.

Does compression affect response speed?

Less data travels faster, especially on slow connections. The small delay from server-side compression is usually more than offset by the reduced transmission time.

Why does curl show the same size with and without the compression flag?

Because --compressed outputs the size of the already-decompressed body. The actual network volume is smaller. Measure the raw stream with a separate script.

Can I compress the body of a POST request?

Yes. The client sets Content-Encoding in the request, but the server must be able to decompress it. Not all servers support this, so check the API documentation.

Does compression work through the Proxeon proxy?

Yes. The proxy passes Accept-Encoding and Content-Encoding headers transparently, so all savings are preserved. That's why it's worth measuring through the proxy—in production conditions.

What if the server doesn't serve compression?

Make sure you're requesting compression and the resource is text. If the server still stays silent, you can't change someone else's server, but you can choose a different endpoint or compress data at your own caching layer.

Should I compress images with Accept-Encoding?

No. Formats like JPEG and WebP are already compressed. Additional compression won't save anything and will just load your CPU.

How do I choose between zstd and brotli?

For downloading, the bandwidth difference is a few percentage points. Include both in Accept-Encoding and let the server decide. If the server supports only one, you'll get that one.

Do I need Vary for client-side work?

As a client, you don't set Vary—the server does. But understanding it helps: it explains why a cache sometimes serves compressed and sometimes uncompressed versions.

Conclusion

You've come all the way from theory to working measurements. Now you understand how compression is negotiated via Accept-Encoding, Content-Encoding, and Vary. You can request compression in curl, Python, and Node.js, and—most importantly—measure the actual network volume before and after. You know why a response sometimes comes back uncompressed and how to diagnose it across three typical causes. You've tackled the pitfalls: double compression, corrupted streams, and manual decompression. And you no longer waste resources compressing images, videos, and archives.

What to do next. Run the measurement script against all the main endpoints in your project. Find the ones where compression isn't enabled and request it explicitly. Keep a log of bytes saved to see the financial benefit of working through the Proxeon proxy. Once you've mastered the basics, move on to mass measurement across a URL list and algorithm priorities with q-values.

Compression is one of the cheapest ways to cut your bandwidth bill. One correctly sent header can reduce data volume severalfold. Now this tool is in your hands, and you can apply it deliberately, with precise numbers at your fingertips.