How We Made NGINX Vary Cache Purges 1000x Faster

If you purge NGINX cache entries selectively, there is a trap hiding behind one innocent response header: Vary. The moment your upstream says Vary: Accept-Encoding (or you enable gzip_vary), NGINX stores one cache file per distinct request header value under the same cache key. An NGINX Vary cache purge therefore has to remove all of those files, not just one.

ngx_cache_purge 2.6.2 fixed the correctness side of this. An exact purge now deletes every stored variant of the key, not just the variant matching the purge request's own headers. Release 2.7.0, out now, fixes the cost side. Here is the story, with benchmarks.

A purge, end to end

A minimal setup with the module loaded. The cached location itself accepts PURGE requests from localhost:

proxy_cache_path /var/cache/nginx/site keys_zone=site:10m;

server {
    listen 80;

    location / {
        proxy_pass        http://backend;
        proxy_cache       site;
        proxy_cache_key   $uri$is_args$args;
        proxy_cache_valid 200 10m;

        # same-location purging: PURGE method on the same URL
        proxy_cache_purge PURGE from 127.0.0.1;
    }
}

Now two different clients request the same URL. The backend answers with Vary: Accept-Encoding, so NGINX stores two cache files for the one key /style.css:

curl -H 'Accept-Encoding: gzip' http://example.com/style.css   # variant 1
curl http://example.com/style.css                              # variant 2

A purge is the same URL with the PURGE method. One request removes the key and every variant, and the Cache-Purge-Result header tells you exactly how many files went away:

curl -X PURGE -i http://127.0.0.1/style.css

HTTP/1.1 200 OK
Content-Type: text/html
Cache-Purge-Result: exact; count=2

<html>...<h1>Successful purge</h1><p>Key : /style.css</p>...</html>

count=2 is both variants. Before 2.6.2 the same request would have removed at most one file - whichever variant matched the purge request's own Accept-Encoding - and typically answered 412 Precondition Failed while stale variants stayed servable.

Why Vary variants are invisible to a purge

NGINX names a variant's cache file by hashing the cache key together with the requesting client's header values. The gzip client above produces one file name; the no-encoding client produces another. The purge request knows the cache key. It cannot know every Accept-Encoding string your clients have ever sent, so it cannot compute the variant file names. NGINX keeps no on-disk index of variants either. (Our proxy cache and microcaching guide covers how these cache files are laid out.)

That leaves exactly one correct way to find the variants: walk the whole cache zone and check each file's embedded KEY: line. This is what 2.6.2 does. It is the same cost class as a wildcard purge: O(every file in the zone) for a single exact purge.

How much does the scan cost?

We measured an NGINX Vary cache purge against zones of 10k, 100k and 1M small cache files. Setup: single worker, Linux ext4 in a container on Apple M2; the benchmark script (bench/variant_scan_bench.sh) ships in the module source, so the numbers are reproducible. "Warm" means the page cache already holds the zone; "cold" is the first purge after the page cache was dropped:

Each warm cell is the median of three trials that agreed within a few percent. The non-Vary control purge stayed at a fraction of a millisecond across every zone size and run. That control is the measurement noise floor, and the effects above sit two to six orders of magnitude over it. These are order-of-magnitude numbers from a laptop-hosted VM, not lab-grade latencies, but the shape is unambiguous.

The scary part is not the number itself. It is that the NGINX worker is blocked for the whole scan. A purge that takes 4.5 seconds means one worker serving nothing for 4.5 seconds. On a CDN-sized zone with a million files, a cold purge parks a worker for minutes.

The fix: a variant index that never guesses

Version 2.7.0 introduces a shared-memory variant index: a map from each cache key to the variant hashes that were stored or served for it. When an exact purge finds the key in the index, it unlinks exactly those files. That is O(number of variants) instead of O(zone).

The interesting part is how it stays correct. An index that might be missing a variant is worse than no index. Trusting it would leave stale content servable after a "successful" purge, which is precisely the bug 2.6.2 eliminated. So the index follows one rule:

An index entry is created only at the moment an exact purge has just removed every file for the key. That is the one state in which "no variants exist" is provably true.

From then on, a header filter records each variant NGINX stores or serves into that entry. Anything that casts doubt drops the entry: an NGINX restart, the entry being evicted under index memory pressure, or more than 64 variants on one key. The next purge then simply falls back to the full scan, which re-seeds the index on completion. Stale entries pointing at files the cache manager already evicted are skipped harmlessly. Correctness is never traded for speed; the worst case is always exactly the 2.6.2 behavior.

In practice that means: the first exact purge of a Vary'd key in an NGINX instance's lifetime scans. Every purge after that is indexed.

Which purges pay what

Not every purge is affected, and the expensive case is narrower than it may sound:

Two practical notes. You can tell which path a purge took from the error log at info level: the scan logs exact purge removed N vary variant(s) and the indexed path appends via variant index. And if a response should never have had variants in the first place, proxy_ignore_headers Vary in the cached location avoids the whole topic - at the price of serving one representation to all clients.

The payoff

Same benchmark, index enabled (the default):

Repeat purges land in the 0.2-2 ms range regardless of zone size. At a million files that is roughly a 1000× improvement over the warm scan, and five orders of magnitude over the cold one. Purge results are identical either way: Cache-Purge-Result: exact; count=N reports the same counts.

Configuration

Nothing to do. The index is on by default with an 8 MB shared memory zone, which tracks tens of thousands of purged keys. One http-level directive controls it:

# default; size it up for very large purge-heavy deployments
cache_purge_variant_index 8m;

# or always use the full scan
cache_purge_variant_index off;

Non-Vary purges never scan and never touch the index. If your upstreams send no Vary header, none of this affects you: purges were and remain sub-millisecond.

Getting it

ngx_cache_purge 2.7.0 is available from the GetPageSpeed repositories for all supported RPM and DEB platforms:

# RHEL-likes
dnf -y install https://extras.getpagespeed.com/release-latest.rpm
dnf -y install nginx-module-cache-purge

# Debian/Ubuntu: see the repo setup instructions, then
apt install nginx-module-cache-purge

Then load the module and reload NGINX:

load_module modules/ngx_http_cache_purge_module.so;

The full changelog, directive reference and the correctness model live on the cache-purge module page.

If you run WordPress behind NGINX, the module pairs well with our WordPress cache purge setup. And if you came here comparing purging options, ngx_cache_purge vs Varnish covers the feature gap this module keeps closing.