HTTPS and the Web Server in Practice: Part 4 — HTTP and CDN Caching: Cache-Control, ETag, proxy_cache, and Cache Invalidation

Caching is the cheapest way to speed up a site and reduce server load: a response that doesn’t need to be regenerated and retransmitted arrives instantly. But caching has a downside — users see old styles after a deploy, a CDN serves one visitor another’s private page, or changes on the site “don’t apply”. Almost all such problems are solved with the correct Cache-Control headers and an understanding of where the copy is stored. A brief overview of caching headers is in the article HTTP, HTTPS and TLS: Part 2 — HTTP Headers: types and functions; here — how to use them.

Where copies are stored

There are usually several caches between the user and the application:

1. Browser cache — personal, only for one user.
2. CDN (Cloudflare, cloud and Russian CDNs) — a shared cache on servers around the world; one copy is served to all visitors.
3. Reverse proxy — Nginx with proxy_cache, Varnish — a shared cache close to the application.
4. Cache inside the application — Redis, memcached, page caches in CMSs. Not related to HTTP headers.

The server sets rules for the first three via the response headers. Shared caches (CDN, proxy) are by default more conservative than the browser, and CDNs have their own settings that can override headers.

How the cache decides what to do

A cached copy can be fresh or stale. A fresh cache is served without asking the server. A stale one is first validated — with a conditional request using If-None-Match (by ETag) or If-Modified-Since (by Last-Modified). If the resource hasn’t changed, the server replies with a short 304 Not Modified without a body, and the cache extends the copy.

How long a copy stays fresh is set by Cache-Control: max-age in seconds. If neither Cache-Control nor Expires is present, caches may heuristically estimate the lifetime — usually 10% of the time since Last-Modified. A file modified a year ago might be considered fresh by the browser for more than a month. Therefore explicit headers are always needed.

Cache-Control directives

For CDNs there is a separate header CDN-Cache-Control (RFC 9213): it applies only to CDNs and is not visible to the browser. This allows keeping a page in the CDN for an hour while not caching it in the browser.

Ready-made strategies

Static assets with a hash in the filename (app.3f9a1c.js, style.8b2e.css — produced by Vite, webpack, Hugo Pipes): when the file changes, its name changes, so the old copy can be stored forever.

Cache-Control: public, max-age=31536000, immutable

HTML pages: links to new versions of static assets are in the HTML, so the HTML must be updated immediately.

Cache-Control: no-cache
ETag: "5f2a-18c4"

The browser stores the page but validates it each time: if nothing changed — 304 without a body.

Static assets without a hash (images from a CMS, favicon.ico, robots.txt): short lifetime plus validation.

Cache-Control: public, max-age=3600, stale-while-revalidate=86400

Personal pages and API responses with user-specific data:

Cache-Control: private, no-store

Public API data that changes frequently (rates, stock counts):

Cache-Control: public, max-age=10, stale-while-revalidate=30

Configuration in Nginx

Headers for static assets and HTML:

location ~* \.(?:js|css|woff2)$ {
    # only for files with a hash in their name
    add_header Cache-Control "public, max-age=31536000, immutable" always;
}

location ~* \.(?:png|jpe?g|webp|avif|svg|ico)$ {
    add_header Cache-Control "public, max-age=86400" always;
}

location / {
    add_header Cache-Control "no-cache" always;
    try_files $uri $uri/ /index.html;
}

Nginx itself serves ETag and Last-Modified for static files and replies 304 to conditional requests. For application responses, ETag should be set by the application.

Cache in Nginx: proxy_cache

Nginx can store application responses itself and serve them without contacting the app:

proxy_cache_path /var/cache/nginx/app levels=1:2 keys_zone=app:50m
                 max_size=2g inactive=7d use_temp_path=off;

server {
    location / {
        proxy_pass http://127.0.0.1:8080;

        proxy_cache app;
        proxy_cache_key "$scheme$host$request_uri";
        proxy_cache_valid 200 301 10m;
        proxy_cache_valid 404 1m;

        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_background_update on;
        proxy_cache_lock on;

        # don't cache for logged-in users
        proxy_cache_bypass $cookie_sessionid;
        proxy_no_cache     $cookie_sessionid;

        add_header X-Cache-Status $upstream_cache_status always;
    }
}

• proxy_cache_use_stale — serve a stale copy when the app errors: the site stays available while the app restarts;
• proxy_cache_background_update and updating — update a stale copy in the background while serving the old one (analogous to stale-while-revalidate);
• proxy_cache_lock — on a miss only one request goes to the application, others wait for its result. Protects against the thundering herd when a popular page falls out of the cache;
• X-Cache-Status will show HIT, MISS, EXPIRED, STALE, UPDATING or BYPASS.

Nginx respects application headers: it will not cache responses with Set-Cookie, with Cache-Control: private, no-store, no-cache, and it treats the Cache-Control: max-age lifetime as higher priority than proxy_cache_valid.

Microcaching — a one-second cache (proxy_cache_valid 200 1s) for public pages under load: with a thousand requests per second only one reaches the application, and data is stale by no more than one second.

CDN

What to know about CDNs:

• CDNs cache by their own rules. Cloudflare by default caches only static files by file extension (.js, .css, images, fonts), but not HTML. To cache pages you need a cache rule (Cache Rules). Other CDNs may, conversely, cache everything allowed by headers;
• cache key — usually the host, path and query string. Parameters like utm_source, fbclid, yclid create thousands of copies of the same page — you should exclude them from the key in the CDN settings;
• cookies. If a response depends on cookies, and the CDN caches it by URL, one visitor may receive another’s page. Personal responses should be private or bypass the cache when a session cookie is present;
• cache status is visible in headers: Cloudflare uses CF-Cache-Status (HIT, MISS, EXPIRED, DYNAMIC — not cached by rules, BYPASS — bypass due to response headers), others use X-Cache, and the Age header shows the age of the copy.

Check:

curl -sI https://example.ru/assets/app.3f9a1c.js | grep -iE 'cache-control|cf-cache-status|x-cache|age|etag'

Repeat the request: with correct configuration the second time you’ll see HIT and a non-zero Age.

Cache purging

• Filenames with a hash — the primary approach for static assets: no need to purge anything, a new version has a new address.
• Version parameter (style.css?v=42) — works if the CDN includes the query string in the key; worse than a hash because the version is updated manually.
• Purge in the CDN — via the dashboard or API, by URLs or entirely. After a deploy it’s useful to purge HTML; purging “everything” after every deploy defeats the purpose of static asset caching.
• Nginx proxy_cache — remove the cache directory or individual files; in the free version of Nginx there is no command to purge by URL, third-party modules or Nginx Plus provide that.
• You can’t clear the browser cache from the server. If you served a file with max-age=31536000 without a hash in the filename, users will see the old version until the TTL expires or they clear their cache. The response header Clear-Site-Data: "cache" clears the site’s cache in the browser, but only when the browser receives a response with it — and the page that itself sits in the cache is not requested from the server. Therefore long lifetimes are only suitable for files with a hash.

Common mistakes

• No Cache-Control at all — browsers cache heuristically, and after a deploy some users see the old layout.
• max-age set to a year for style.css without a hash in the filename — changes don’t reach returning visitors.
• HTML cached for a long time — the page links to files from the old build and the site “breaks” for some users.
• A personal page without private cached by the CDN as HTML — data leaks between users.
• no-cache used instead of no-store where data must not be saved to disk at all.
• Purging the entire CDN cache on every deploy — after deploy all requests hit the origin at once.
• Debugging “why didn’t it update” with the browser cache enabled — check with curl or use Disable cache in DevTools.