Cache & CDN Invalidation Testing

When map tiles, vector style bundles, sprite sheets, or glyph PBFs fail to invalidate predictably, a visual regression run captures whatever the edge happened to be holding — a stale basemap under a fresh overlay, a half-rotated style, or a coordinate projection that no longer matches the live data. The diff then fails for a reason that has nothing to do with cartography, and the report buries a genuine regression under cache noise. Cache and CDN invalidation testing is the discipline of proving, before any screenshot is taken, that every tier of the caching hierarchy — browser HTTP cache, service worker, CDN edge, origin shield, and origin tile server — has converged on the exact asset versions the test expects.

This page extends the masking and stability work described in Dynamic Element Masking & UI Stability: masking neutralizes volatile on-screen elements, while cache invalidation neutralizes volatile upstream state, and a baseline is only trustworthy when both are controlled.

What Cache Invalidation Means for Map Assets

In a web mapping context, “the asset” is rarely a single file. A single rendered frame is composed from a raster or vector tile grid, a style.json, one or more sprite atlases (sprite.png + sprite.json), and glyph ranges fetched per font stack. Each of these is independently cacheable and independently versionable, so invalidation is a set operation: the test must assert that the whole composition is coherent, not that one URL returned 200.

The HTTP contract is governed by standard freshness and revalidation semantics. The MDN Web Docs on HTTP Caching define how Cache-Control: max-age, s-maxage, stale-while-revalidate, ETag, and Last-Modified interact across browsers and shared caches. Two properties matter most for deterministic capture:

  • Strong validators. An ETag derived from a content hash lets every tier answer a conditional request with 304 Not Modified for unchanged grids and force a full 200 OK only when bytes actually changed. Weak validators (W/"…") are not sufficient when subpixel rendering is being diffed.
  • Immutability with fingerprinting. Append a content hash to the request path (style.7f3a9c.json, sprite.7f3a9c.png) and serve it with Cache-Control: public, max-age=31536000, immutable. A new render is a new URL, so invalidation collapses into deployment rather than purging.

The OGC Tile Map Service and the XYZ slippy-map convention both key a tile by z/x/y, which means the content of z/x/y changes whenever the underlying data or style changes, but the URL does not. That mismatch — a mutable resource at a stable URL — is the root cause of nearly every stale-tile false positive, and the architecture below exists to resolve it. The deeper version-keying model lives in Baseline Management for Tile Servers; the techniques here are how you prove the live edge agrees with that key at capture time.

Architecture of the Caching Hierarchy

A request for a tile traverses, in order: the browser HTTP cache, any registered service worker Cache Storage, the CDN edge node nearest the runner, an optional origin shield, and finally the origin tile server. Each tier can independently hold a stale copy, and a purge that reaches the edge but not the service worker still yields a stale render.

The five caching tiers a tile request traverses A request descends through five independently cacheable tiers — browser HTTP cache, service worker Cache Storage, CDN edge node, origin shield, and origin tile server. Each row names the validator that governs the hop and the stale-render failure mode that occurs if that tier is not invalidated before capture. TIER VALIDATOR AT THIS HOP IF NOT INVALIDATED 1 Browser HTTP cache closest to the pixels max-age / heuristic freshness strong ETag for 304s serves disk copy, never reaches network 2 Service worker Cache Storage, below the network versioned cache name tiles-v<hash> returns stale tile after a clean purge 3 CDN edge node nearest the runner s-maxage + Cache-Tag surrogate-key purge partial propagation across PoPs 4 Origin shield serializes the purge single warm revalidation one conditional GET origin stampede, inconsistent state 5 Origin tile server source of truth content-hash ETag mutable z/x/y, stable URL new bytes at an unchanged address Any one tier left stale corrupts the captured frame — invalidation is a set operation across all five.

The recommended structural approach rests on three load-bearing choices:

  1. Surrogate-key tagging at the edge. Tag every tile and style response with cache tags (Fastly surrogate keys, Cloudflare Cache-Tag, CloudFront via invalidation paths) such as style:7f3a9c, layer:roads, and z:14. Purging by tag clears exactly the affected composition without a wildcard purge that would cold-start the entire grid and inflate capture latency.
  2. Immutable fingerprints for everything addressable. Styles, sprites, and glyphs carry a content hash in the path. Only the raster/vector tile endpoints stay at stable z/x/y URLs, and those are the only resources that ever need an active purge.
  3. Origin shielding to serialize purges. A shield tier absorbs the purge once and revalidates downstream edges from a single warm copy, so a distributed test fleet does not stampede the origin and observe inconsistent intermediate state.

The service worker tier deserves special attention because it sits below the network from the page’s perspective and will happily serve a cached tile even after every server-side tier is purged. A map application’s service worker must either version its cache names (tiles-v7f3a9c) and delete stale buckets on activate, or expose a test-time message channel that flushes Cache Storage on demand. Without that, you are diffing the contents of the client’s disk, not the deployment.

Step-by-Step Invalidation Verification

The verification harness runs as a pre-flight stage in the deployment pipeline, before the visual suite is allowed to capture. It snapshots current validators, executes the purge, confirms propagation across edges, and only then green-lights capture.

Pre-flight invalidation gate sequence A sequence between five actors: the CI pipeline, the CDN edge, the origin shield, the origin tile server, and the visual tests. The pipeline snapshots validators, purges by surrogate key, the edge invalidates the tag and revalidates the shield against origin, the pipeline polls each edge region for a MISS or PURGED status, asserts the new hash on a warming GET, and only then releases the capture gate to the visual tests. CI pipeline CDN edge Origin shield Origin tiles Visual tests 1 · snapshot ETag / Cache-Control 2 · purge by surrogate key style:hash 3 · invalidate tag → revalidate 4 · conditional GET 5 · HEAD from each edge region CI (return) --> 6 · MISS / PURGED per PoP 7 · GET — assert new hash + max-age CI (return) --> 8 · 200, ETag = deployment 9 · release gate — begin capture

The numbered procedure the diagram encodes:

  1. Snapshot current validators. For every critical asset — style.json, each sprite sheet, and a representative tile sweep — record the live ETag, Last-Modified, and Cache-Control so the post-purge state can be diffed against a known baseline.

    for url in \
  "$CDN/styles/main/style.json" \
  "$CDN/sprites/main/sprite.png" \
  "$CDN/tiles/14/8645/5293.pbf"; do
  curl -sI "$url" \
    | grep -iE '^(etag|last-modified|cache-control|cf-cache-status|x-cache):' \
    | sed "s#^#$url  #"
done

  2. Execute a tag-scoped purge. Trigger invalidation through the CDN API using the surrogate key for the deployed style hash, never a wildcard path, to keep the purge blast radius minimal.

    # Cloudflare: purge by cache tag rather than full-zone
curl -sX POST \
  "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
  -H "Authorization: Bearer $CF_API_TOKEN" \
  -H "Content-Type: application/json" \
  --data "{\"tags\":[\"style:${STYLE_HASH}\",\"sprite:${STYLE_HASH}\"]}"

  3. Confirm propagation from every edge region. Dispatch concurrent HEAD requests from geographically distributed runners and assert each one reports a purged/miss state before any GET warms it.

    // Assert every edge reports a purge before capture is allowed.
const regions = ['iad', 'lhr', 'fra', 'syd', 'gru'];
const results = await Promise.all(regions.map(async (pop) => {
  const res = await fetch(`https://${pop}.edge.example.com/styles/main/style.json`, {
    method: 'HEAD',
    cache: 'no-store',
  });
  const status = res.headers.get('cf-cache-status') ?? res.headers.get('x-cache');
  return { pop, status, etag: res.headers.get('etag') };
}));
const stale = results.filter((r) => !/MISS|PURGED|EXPIRED/i.test(r.status ?? ''));
if (stale.length) throw new Error(`Stale edges: ${JSON.stringify(stale)}`);

  4. Assert freshness on the warming GET. Re-request each asset and verify it now carries the deployment’s content hash and the expected max-age, proving the new version — not a re-warmed stale copy — is what the next capture will see.

  5. Flush the client tiers in the test browser. Bypass the browser HTTP cache and block service workers so the page composes its frame from freshly fetched assets rather than disk. This is the bridge between server-side invalidation and what the comparator actually photographs.

    // Playwright: deterministic client-side cache state for capture.
const browser = await chromium.launch({
  args: ['--disable-application-cache', '--disk-cache-size=0', '--no-sandbox'],
});
const context = await browser.newContext({
  serviceWorkers: 'block',
  bypassCSP: true,
});
// Belt-and-braces: force-revalidate every request during capture.
await context.route('**/*', (route) => {
  const headers = { ...route.request().headers(), 'cache-control': 'no-cache' };
  route.continue({ headers });
});

  6. Release the capture gate. Only after steps 3 and 4 pass for the full asset set does the harness hand control to the screenshot stage. Capture synchronization from that point — waiting for tiles to hydrate — is owned by Handling Async Tile Loading, and the broader capture pipeline by Screenshot Capture, Sync & Comparison Logic.

Webhook-driven purges should fire only after origin deployment and asset fingerprinting both succeed; a purge that races ahead of the origin leaves edges revalidating against the old origin and re-caching stale bytes — the single most common way an “invalidation” makes the problem worse. Standardize the header directives across environments against the Cloudflare Cache Control Guidelines so staging and production cannot drift apart.

Cross-Browser & Cross-Environment Considerations

Cache heuristics are not uniform across engines, and a purge strategy validated only on Chromium will still produce stale renders on WebKit or Firefox.

  • Chromium honours --disk-cache-size=0 and the --disable-application-cache flag, and respects serviceWorkers: 'block' at the context level. Its heuristic freshness (caching a no-Cache-Control response for ~10% of the Last-Modified age) means tile endpoints must send explicit directives.
  • WebKit (Safari / Playwright webkit) ignores several Chromium-specific flags; the reliable lever is request-level Cache-Control: no-cache injected via routing, plus cache: 'no-store' on programmatic fetch. WebKit is also stricter about partitioned (per-origin) Cache Storage, so a shared service worker cache across subdomains will not behave as on Chromium.
  • Firefox (Gecko) applies its own heuristic freshness window and keeps a separate in-memory cache that survives soft reloads; launch with browser.cache.disk.enable=false and browser.cache.memory.enable=false preferences for a clean capture environment.

In containerized CI, pin the browser image (the same pinned-image discipline that stabilizes fonts and GL drivers) so that a cache-behaviour change in a browser point release cannot silently alter results. Distributed runners should resolve the CDN through deterministic DNS or a fixed edge PoP per shard; otherwise two shards hit two edges in different purge states and disagree. Where transient UI chrome appears during the cold-cache reflow — spinners, skeleton tiles, progress bars — suppress it with Animation & Transition Suppression and exclude the interactive layer per Interactive Overlay Masking Rules so a slow first paint after a purge does not register as a regression.

Threshold & Parameter Reference

These are the configurable values that govern an invalidation gate. Treat them as starting points and tighten per CDN SLA. Propagation budget can be expressed as — the capture stage must not begin before elapses or the runner risks photographing a mid-propagation edge.

Parameter Recommended value Rationale
Edge purge propagation budget < 500 ms (enterprise CDN) SLA ceiling before capture is allowed to start
Propagation poll backoff 100 ms base, ×2, max 5 tries Exponential backoff with jitter absorbs uneven edge timing
Tile Cache-Control (active testing) no-cache, must-revalidate Forces conditional revalidation on every capture
Fingerprinted asset max-age 31536000, immutable Styles/sprites/glyphs never need active purge
Background terrain stale-while-revalidate 60 Tolerable only for non-critical, non-diffed layers
Edge regions polled per gate ≥ 5 (multi-continent) Detects partial-propagation before capture
Capture retry on propagation timeout 3, exponential backoff Avoids flaky failures from a single slow edge
Validator type strong ETag (content hash) Weak validators allow subpixel-divergent 304s

For non-critical background layers (base terrain, hillshade), a short stale-while-revalidate is acceptable because those layers are typically masked or low-weighted in the diff. Foreground vector layers under active test must use no-cache or must-revalidate so the comparator never sees a revalidation-in-progress frame. How much per-pixel tolerance to allow once the right bytes are confirmed is a separate calibration covered by Dynamic Threshold Configuration.

Common Pitfalls

Purge fired before origin deployment finished

Root cause: the invalidation webhook triggered on commit or build start rather than on a confirmed origin deploy, so edges revalidated against the old origin and re-cached stale bytes. Diagnose: compare the ETag returned by the warming GET against the deployed artifact hash; a mismatch on a freshly purged URL confirms the race. Fix: gate the purge webhook on a successful origin health check and asset-fingerprint step, and re-poll edges after the purge rather than assuming instant convergence.

Service worker serves stale tiles after a clean CDN purge

Root cause: the application service worker holds tiles in versionless Cache Storage and is not flushed by the test, so the page never reaches the network. Diagnose: in the failing run, log caches.keys() and the from-service-worker flag on tile responses; cached hits with the old hash confirm it. Fix: version cache names by style hash and delete stale buckets on activate, or block service workers entirely in the capture context with serviceWorkers: 'block'.

Wildcard purge cold-starts the grid and inflates capture latency

Root cause: purging by path wildcard (/tiles/*) evicts the entire grid, so the first capture pays a full origin round-trip per tile and times out or photographs a half-filled grid. Diagnose: correlate a spike in x-cache: MISS density and tile-load duration with the purge event. Fix: purge by surrogate key scoped to the changed style:hash/layer:id and let an origin shield re-warm edges from one copy before the gate opens.

Heuristic freshness caches a directive-less tile endpoint

Root cause: a tile response with no Cache-Control is cached by the browser for a fraction of its Last-Modified age, so a “no-cache” intent silently becomes minutes of staleness. Diagnose: inspect the tile response headers; an absent Cache-Control with a present Last-Modified is the signature. Fix: send explicit Cache-Control on every tile and style response and inject request-level no-cache during capture for engines that ignore launch flags.

Edges disagree because shards hit different PoPs mid-propagation

Root cause: two parallel shards resolved the CDN to two different edge nodes that were in different purge states, producing one pass and one fail for the same view. Diagnose: log the resolved edge PoP and cf-cache-status per shard; divergent PoPs with divergent statuses confirm it. Fix: poll all target regions in the gate before capture, or pin each shard to a fixed PoP and only release once every polled edge reports a purge.

Cache misbehaviour that survives all of the above usually manifests as residual rendering noise rather than a clean stale frame; the preprocessing filters in Noise Reduction for Map Artifacts are the last line of defence, but they are no substitute for a correct purge.

← Back to Dynamic Element Masking & UI Stability