Storing Map Baselines in S3 with Git LFS Manifests

A visual regression suite for a slippy map accumulates baselines fast: a few zoom levels across a handful of viewports, per browser engine, at one or two device pixel ratios, and you are already holding hundreds of PNGs weighing tens of megabytes. Commit those blobs to Git and every clone drags the full history of every re-blessed tile forever; the repository balloons, git gc crawls, and code review drowns in binary diffs no human can read. This page gives a concrete procedure for the opposite arrangement: push the binary tile baselines to S3 (or GCS) content-addressed by a hash of the style and capture parameters, commit only a small JSON manifest per baseline, and have CI pull back exactly the byte-identical set a given commit expects.

This is one task inside Baseline Management for Tile Servers, the guide covering how deterministic tile captures are stored and versioned. It sits under the broader discipline in Web Map Visual Testing Fundamentals & Toolchains, and it assumes you already produce reproducible frames — an unstable renderer produces a different hash on every run and defeats content addressing before it starts.

Prerequisites

Step-by-step procedure

1. Structure the manifest

A manifest is a small, human-diffable JSON file that records everything needed to identify one baseline and locate its bytes. Commit these; never commit the PNG. Each manifest names the capture parameters, the content hash of the image, and the storage key derived from it. Keep one manifest per baseline under baselines/ so a code review shows exactly which baselines a pull request re-blesses.

{
  "id": "home-z12-chromium-dpr1",
  "viewport": { "width": 1024, "height": 768 },
  "dpr": 1,
  "engine": "chromium-swiftshader",
  "camera": { "center": [-0.1278, 51.5074], "zoom": 12, "bearing": 0, "pitch": 0 },
  "style_hash": "a4f9c1e7",
  "image_sha256": "9b2c...e1",
  "bytes": 41822,
  "storage_key": "baselines/a4f9c1e7/9b2c...e1.png"
}

The image_sha256 is the integrity check; the storage_key is where the bytes live. Recording style_hash separately lets you group and roll back a whole layer’s baselines without reading every image hash.

2. Content-address by style hash

Derive the storage key from content, not from a mutable name. Hash the style.json (plus DPR and engine, which change the pixels) into a short style_hash, and hash the PNG bytes into image_sha256. The object key is baselines/<style_hash>/<image_sha256>.png. Because the key is a pure function of the inputs, re-running an unchanged capture writes to the same key — uploads become idempotent, and identical baselines across tests deduplicate automatically.

import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";

const sha256 = (buf) => createHash("sha256").update(buf).digest("hex");

// style_hash folds the style definition and the pixel-affecting knobs together.
export function styleHash(styleJson, { dpr, engine }) {
  const canonical = JSON.stringify(styleJson) + `|dpr=${dpr}|engine=${engine}`;
  return sha256(Buffer.from(canonical)).slice(0, 8);
}

export function storageKey(styleJson, opts, pngPath) {
  const sh = styleHash(styleJson, opts);
  const imgHash = sha256(readFileSync(pngPath));
  return { style_hash: sh, image_sha256: imgHash, key: `baselines/${sh}/${imgHash}.png` };
}

Content addressing means a baseline is immutable: a change to the style produces a new style_hash and therefore a new path, so old baselines are never mutated in place — they are simply no longer referenced by the current manifests.

3. Upload to S3 with a lifecycle policy

Sync new blobs to the bucket. Because keys are content hashes, aws s3 cp with a guard is safe to re-run; you never overwrite an existing object with different bytes. Set a lifecycle policy so orphaned baselines — objects no manifest references after re-blessing — are transitioned to cheaper storage and eventually expired, rather than accumulating cost forever.

#!/usr/bin/env bash
# baselines:push — upload any PNG referenced by a manifest but absent from S3.
set -euo pipefail
BUCKET="s3://mapviz-baselines"

for manifest in baselines/*.json; do
  key=$(jq -r '.storage_key' "$manifest")
  local_png="artifacts/$(jq -r '.id' "$manifest").png"
  if ! aws s3api head-object --bucket mapviz-baselines --key "$key" >/dev/null 2>&1; then
    aws s3 cp "$local_png" "$BUCKET/$key" --content-type image/png
  fi
done

A matching lifecycle rule keeps storage bounded. Baselines under baselines/ that stop being referenced after a re-bless are the only ones that age out, and versioning protects against an accidental delete:

{
  "Rules": [
    {
      "ID": "expire-orphaned-baselines",
      "Filter": { "Prefix": "baselines/" },
      "Status": "Enabled",
      "Transitions": [{ "Days": 30, "StorageClass": "STANDARD_IA" }],
      "NoncurrentVersionExpiration": { "NoncurrentDays": 90 }
    }
  ]
}

4. Write a baselines:pull script for CI

CI must fetch exactly the baseline set the checked-out commit expects — no more, no less. Read every manifest, pull each storage_key into a local baseline/ directory, and skip any object already cached from a previous run keyed by its hash. This is the step the visual-test job calls before comparing, exactly where the parent guide’s GitHub Actions job runs npm run baselines:pull.

// baselines:pull — hydrate baseline/ from S3 using the committed manifests.
import { readdirSync, readFileSync, existsSync, mkdirSync } from "node:fs";
import { execFileSync } from "node:child_process";

const BUCKET = "s3://mapviz-baselines";
mkdirSync("baseline", { recursive: true });

for (const file of readdirSync("baselines").filter((f) => f.endsWith(".json"))) {
  const m = JSON.parse(readFileSync(`baselines/${file}`, "utf8"));
  const dest = `baseline/${m.id}.png`;
  if (existsSync(dest)) continue; // cache hit — bytes are immutable by hash
  execFileSync("aws", ["s3", "cp", `${BUCKET}/${m.storage_key}`, dest], {
    stdio: "inherit",
  });
}

Because keys are content-addressed, the runner’s baseline cache can be keyed on image_sha256 and reused across builds — a baseline never changes under a fixed hash, so a cache hit is always correct.

Manifests in Git resolve to content-addressed objects in S3 On the left, Git version control holds three small JSON manifests, each recording a style hash, an image SHA-256, and a storage key; no binary PNGs are committed. Each manifest points by hash to an object in the S3 bucket on the right, whose key is baselines slash style hash slash image hash dot png. Two manifests that share a style hash and image hash resolve to the same object, showing deduplication. A lifecycle policy box beneath the bucket transitions orphaned objects to infrequent-access storage and expires old noncurrent versions. CI pulls objects back by key into a local baseline directory using the manifests. Commit manifests, address bytes by hash Git — manifests only home-z12.json a4f9c1e7 · 9b2c…e1 home-z12-webkit.json a4f9c1e7 · 9b2c…e1 roads-z14.json c07d55a1 · 3f18…9d S3 bucket — content-addressed baselines/a4f9c1e7/ 9b2c…e1.png baselines/c07d55a1/ 3f18…9d.png Lifecycle: orphaned objects → IA storage, noncurrent versions expire after 90 days resolve by hash

5. Verify integrity by hash

Never trust a downloaded baseline blindly — a truncated transfer or a bucket mishap must fail loudly, not silently pass a comparison against corrupt pixels. After pulling, re-hash each local PNG and assert it matches the image_sha256 in its manifest. A mismatch means the bytes on disk are not the bytes the commit blessed.

import { readFileSync, readdirSync } from "node:fs";
import { createHash } from "node:crypto";

let failed = 0;
for (const file of readdirSync("baselines").filter((f) => f.endsWith(".json"))) {
  const m = JSON.parse(readFileSync(`baselines/${file}`, "utf8"));
  const actual = createHash("sha256").update(readFileSync(`baseline/${m.id}.png`)).digest("hex");
  if (actual !== m.image_sha256) {
    console.error(`integrity fail: ${m.id} expected ${m.image_sha256} got ${actual}`);
    failed++;
  }
}
if (failed) process.exit(1);

The probability that a corrupt transfer collides with the recorded SHA-256 is — effectively zero — so a passing check is a hard guarantee the baseline is exactly what was uploaded.

6. Roll back one layer

When a style regression slips into a baseline, content addressing makes rollback surgical. Because each baseline is keyed by its style_hash, you revert only the manifests carrying the bad hash — the old objects still exist in S3 (nothing was overwritten), so restoring the previous manifest restores the previous pixels for that layer alone, leaving every other baseline untouched.

# Roll back only the baselines produced by the regressed style hash.
git checkout HEAD~1 -- $(grep -rl '"style_hash": "a4f9c1e7"' baselines/)
npm run baselines:pull   # re-hydrate the reverted keys from S3

This is the granular rollback the parent guide describes: re-bless one layer, not the whole grid.

Verification

Confirm the pipeline is sound before relying on it as a CI gate:

Troubleshooting

Symptom Likely cause Fix
CI pull 404s on a key that exists locally Blob was captured but never pushed, or lifecycle expired an orphan still referenced Run baselines:push before merge; scope the lifecycle rule so referenced objects never expire
Integrity check fails after pull Truncated S3 transfer or a manifest edited without re-uploading the matching PNG Re-run the pull; if it persists, re-capture and re-push so image_sha256 and the object agree
Manifest diff is noisy on every run Style serialized non-canonically, so style_hash churns on unchanged styles Canonicalize style.json (sorted keys) before hashing so identical styles produce identical hashes

Frequently asked questions

Why content-address baselines instead of naming them by test id?

A content hash makes each baseline immutable and self-verifying: the same style and capture parameters always resolve to the same key, so uploads are idempotent, identical baselines deduplicate, and a corrupt download is caught by re-hashing. A mutable name like home-z12.png gets silently overwritten when a style changes, erasing the ability to roll back to the exact prior pixels.

Do I still need Git LFS if the bytes live in S3?

Not strictly — a bare S3 reference in the manifest is enough, and it avoids LFS bandwidth quotas. Git LFS is useful when you want a tracked pointer that travels with the checkout and integrates with existing git lfs pull tooling; in that case store the LFS pointer alongside the manifest and let LFS back onto the same bucket. Most teams find the JSON manifest plus a baselines:pull script simpler than running LFS.

How do I keep S3 storage from growing without bound?

Combine content addressing with a lifecycle policy. Because re-blessing a layer writes new keys and leaves old ones unreferenced, a rule that transitions orphaned baselines/ objects to infrequent-access storage and expires noncurrent versions after a retention window bounds cost. Enable bucket versioning so the policy can expire old versions safely without risking an accidental permanent delete.

What exactly should feed the style hash?

Everything that changes the rendered pixels: the canonicalized style.json, the device pixel ratio, and the browser engine. Leave out anything that does not affect output, such as the test id or a timestamp, or the hash will churn and defeat deduplication. Serialize the style with sorted keys so a semantically identical style always hashes the same.