Per-Zoom Diff Threshold Tables for Slippy Maps

A single global tolerance is wrong for a slippy map because the thing being measured changes character with every zoom level. At zoom 2 the frame is mostly ocean and coastline — a handful of high-contrast edges over a flat fill — so even a two-pixel shift in a country border is a large fraction of the differing content and should fail loudly. At zoom 16 the same frame is packed with building footprints, road casings, and dozens of anti-aliased labels, where the identical comparator will report thousands of legitimately-varying edge pixels between two correct runs. Apply the strict low-zoom number at high zoom and every test flakes red; apply the loose high-zoom number at low zoom and a deleted coastline sails through green. This page builds a per-zoom tolerance table — a pixel threshold, a maximum changed-pixel ratio, and an SSIM floor for each zoom band — and wires it so the harness selects the right row at capture time.

This is one task inside Dynamic Threshold Configuration, the guide that explains why one global tolerance cannot serve a slippy map. It sits under the broader pipeline described in Screenshot Capture, Sync & Comparison Logic, and it assumes deterministic capture is already solved — the frames going into the diff are hydrated and the camera is locked, so the only variable this guide tunes is how the diff stage scores the frame at each zoom.

Prerequisites #

• A slippy-map renderer you can drive headlessly (MapLibre GL JS, Mapbox GL JS, OpenLayers, or Leaflet) with the instance exposed on window.__MAP__ so the harness can read map.getZoom().
• Playwright 1.40+ (or Puppeteer) with a pinned software GL backend, so anti-aliasing is identical across runners and the thresholds you calibrate are not chasing GPU variance.
• A pixel comparator that returns a raw changed-pixel count and accepts a per-pixel threshold — pixelmatch 5.x, odiff, or resemble.js.
• An SSIM implementation (ssim.js, image-ssim) for the structural floor that backstops the pixel gate.
• A representative capture set spanning zoom 2 through 18 for the styles you ship, so the starting numbers can be validated against real diffs rather than guessed.

Step-by-step procedure #

1. Bucket the zoom range into content-homogeneous bands #

Tuning a threshold for all 19 discrete zoom levels is unnecessary and unmaintainable: adjacent levels render nearly the same content mix, so they share tolerances. Collapse the range into four bands whose members are visually homogeneous — sparse overview, regional, street, and detail — and treat each band as one tolerance regime. The band edges below follow where slippy-map content density changes character rather than being evenly spaced.

// Four content-homogeneous zoom bands for a standard XYZ/WMTS pyramid.
const ZOOM_BANDS = [
  { id: 'overview', min: 0,  max: 4,  label: 'ocean, coastlines, country fills' },
  { id: 'regional', min: 5,  max: 9,  label: 'roads, region labels, water bodies' },
  { id: 'street',   min: 10, max: 13, label: 'street grid, town labels, land use' },
  { id: 'detail',   min: 14, max: 18, label: 'buildings, dense labels, POI icons' }
];

2. Pick a starting tolerance per band and justify it by content #

Each band gets three knobs. The pixel threshold is the per-pixel color-distance below which two pixels count as equal — lower is stricter. The max changed ratio is the fraction of frame pixels allowed to differ before the test fails. The SSIM floor is the minimum structural-similarity score the frame must clear. Sparse bands earn a tight per-pixel threshold and a small changed ratio because any real difference is a large share of the little content present. Dense bands need a looser per-pixel threshold and a larger changed ratio to absorb the anti-aliased label halos covered in configuring pixel diff thresholds for anti-aliased map labels, while the SSIM floor stays high so a structural regression still trips the gate even when the pixel budget is generous.

The changed ratio $\rho$ is the raw comparator count normalized by frame area:

$$\rho =\frac{n_{\text{changed}}}{W\cdot H},\text{fail if }\rho >{\rho }_{\text{band}}\vee \text{SSIM}<s_{\text{band}}$$

Here are defensible starting values for a DPR-1 basemap at 1024×768. These are calibration seeds, not law — step 4 validates them.

3. Express the table as a config-as-code object #

Keep the table as versioned code beside the tests, not as magic numbers scattered through spec files. A single frozen object is diffable in review, importable everywhere, and — critically — carries a styleVersion so a threshold change and the style change that justified it land in the same commit.

// tolerance-table.js — versioned beside the visual specs.
export const TOLERANCE_TABLE = Object.freeze({
  styleVersion: 'basemap-v7.2.0',
  dpr: 1,
  bands: {
    overview: { zoom: [0, 4],   pixelThreshold: 0.05, maxChangedRatio: 0.002, ssimFloor: 0.995 },
    regional: { zoom: [5, 9],   pixelThreshold: 0.08, maxChangedRatio: 0.006, ssimFloor: 0.990 },
    street:   { zoom: [10, 13], pixelThreshold: 0.10, maxChangedRatio: 0.015, ssimFloor: 0.985 },
    detail:   { zoom: [14, 18], pixelThreshold: 0.12, maxChangedRatio: 0.030, ssimFloor: 0.980 }
  }
});

4. Select the band at capture time from the live zoom #

Read the actual zoom off the locked map instance rather than trusting the test’s declared intent — a flyTo that was clamped by maxZoom, or a fractional zoom that never settled on an integer, will otherwise pick the wrong row. Round to the rendered integer level, then resolve the band whose range contains it.

function selectBand(zoom, table) {
  const z = Math.round(zoom);
  const entry = Object.entries(table.bands)
    .find(([, b]) => z >= b.zoom[0] && z <= b.zoom[1]);
  if (!entry) throw new Error(`zoom ${z} falls outside every band — extend the table`);
  return { id: entry[0], ...entry[1] };
}

// In the test, read the live zoom from the pinned camera:
const zoom = await page.evaluate(() => window.__MAP__.getZoom());
const band = selectBand(zoom, TOLERANCE_TABLE);

5. Score the diff against the selected band’s three gates #

Feed the band’s pixelThreshold straight into the comparator, then apply both the changed-ratio ceiling and the SSIM floor. A frame passes only if it clears both — the pixel budget catches localized breakage, the SSIM floor catches structural drift that stays under the pixel count.

import pixelmatch from 'pixelmatch';
import { ssim } from 'ssim.js';

function scoreFrame(baseline, candidate, band, { width, height }) {
  const changed = pixelmatch(
    baseline.data, candidate.data, null, width, height,
    { threshold: band.pixelThreshold, includeAA: false }
  );
  const ratio = changed / (width * height);
  const { mssim } = ssim(baseline, candidate);
  return {
    band: band.id,
    pass: ratio <= band.maxChangedRatio && mssim >= band.ssimFloor,
    ratio, mssim
  };
}

6. Validate the seeds against real diffs, then version the table #

Numbers picked from reasoning are a starting point; the table is only trustworthy once it has been run against known-good and known-bad frames at every band. Capture a stability set (same viewport, no change) and confirm each band passes with headroom, then inject a real regression — a deleted layer, a shifted label, a recolored fill — and confirm the same band fails. Tighten any band whose stability margin is thin and loosen any that flags noise. Then bump styleVersion in lockstep with the basemap so the calibration and the style it was calibrated against never drift apart. The SSIM floor choices here share their reasoning with tuning SSIM thresholds for vector basemap diffs in the broader diff algorithm tuning for cartography area.

// Assert every band both passes clean frames and fails a seeded regression.
for (const id of Object.keys(TOLERANCE_TABLE.bands)) {
  const clean = scoreFrame(base[id], base[id], TOLERANCE_TABLE.bands[id], dims);
  const broken = scoreFrame(base[id], regressed[id], TOLERANCE_TABLE.bands[id], dims);
  expect(clean.pass).toBe(true);    // no false positive on identical frames
  expect(broken.pass).toBe(false);  // real regression is caught in this band
}

The relationship the table encodes — tolerance rising as content density rises across the four bands — is easiest to read as a curve.

Verification #

Confirm the table actually discriminates before you trust it as a gate:

• Band coverage. Run selectBand across zoom 0–18; every integer level resolves to exactly one band and none throws. A gap means the table needs another row.
• No false positives. Capture each band’s viewport eight times with no change; all eight clear both the changed-ratio ceiling and the SSIM floor with visible headroom.
• Real regression caught. Seed a deleted layer at overview and a shifted label at detail; both fail in their own band, proving the loose detail budget did not swallow a genuine fault.
• Version lock. The styleVersion in the table matches the basemap the baselines were shot against; a mismatch means the calibration is stale.

If a clean run flakes at detail but the same content passes at street, the detail band is too strict — loosen its changed ratio, not the global one.

Troubleshooting #

Frequently asked questions #

Why four bands instead of a threshold per zoom level?

Adjacent zoom levels render almost the same content mix, so a per-level table is 19 rows that mostly repeat — more surface to drift and to review for no added precision. Four bands (0–4, 5–9, 10–13, 14–18) each cover a content-homogeneous regime, which is the granularity at which the tolerance actually needs to change. If one band proves too coarse — say detail spans both sparse suburbs and dense downtown — split that band rather than exploding the whole table.

Should the SSIM floor also loosen as zoom increases?

Only slightly. The changed-ratio ceiling does most of the work of absorbing dense-content noise; the SSIM floor stays high across all bands because structural similarity is far less sensitive to anti-aliasing halos than a raw pixel count, so a small drop (0.995 down to 0.980) is enough. Keeping the floor high is what lets you hand the detail band a generous pixel budget without letting a real structural regression slip through under it.

How do I keep the table from drifting away from the style?

Carry a styleVersion field inside the frozen table object and bump it in the same commit that changes the basemap style. Because the table is config-as-code beside the specs, a reviewer sees the threshold change and the style change together, and the verification step fails loudly if the baselines were shot against a different version than the table names.

What changes if I capture at a retina device pixel ratio?

DPR-2 doubles the pixel count and the anti-aliased edge surface, so a changed ratio calibrated at DPR-1 will over-report. Keep a separate table per DPR and select on the runner’s actual device pixel ratio rather than reusing one set of numbers — the same rule the parent guide applies to every threshold, not just per-zoom ones.

Related #

• Up to the parent guide Dynamic Threshold Configuration, and the grandparent pipeline Screenshot Capture, Sync & Comparison Logic.
• Configuring pixel diff thresholds for anti-aliased map labels — the label-halo tolerance that the dense bands lean on.
• Tuning SSIM thresholds for vector basemap diffs — how the per-band SSIM floors are chosen.
• Diff Algorithm Tuning for Cartography — the wider set of comparator choices these thresholds plug into.