Pruning Stale Map Baselines Without Losing History

A baseline corpus grows with every axis and every re-bless, and after a year of a healthy suite it is large enough that someone proposes deleting the old ones. That proposal is usually implemented as a rule about age, which is the one criterion guaranteed to delete the artifact a bisect will need — because the whole point of an old baseline is that it is old. The useful criterion is reachability: whether any reader can still exist for it. This procedure prunes on that basis, in a way that is reviewable and revertible.

This is a task within Baseline Management for Tile Servers, under Web Map Visual Testing Fundamentals & Toolchains. It assumes the manifest-plus-object-storage split described in Storing Map Baselines in S3 with Git LFS Manifests.

Prerequisites

Reachability, not age

What a baseline is still needed for, ranked by how likely anyone is to read it Baselines are grouped by who might still read them and how soon. A golden for an active branch is read on every run and cannot be pruned. A superseded golden from the last ninety days is read only by a bisect and only if someone runs one. A golden for a branch that was deleted has no reader at all unless the branch is restored. A golden whose key names an engine or library version no longer in the matrix has no reader by construction. A caption gives the pruning rule that follows: prune by whether a reader can exist, not by age, because age alone deletes the one artifact a bisect needed. Prune by whether a reader can exist, not by age current golden, active branch read every run — never prune the reference the gate compares against superseded, last 90 days read by a bisect — demote, keep the reason a regression can be dated branch deleted no reader unless restored safe to prune after a grace period key names an axis no longer in the matrix no reader by construction

The four groups are produced by joining three things the repository already has: the manifest entries, the list of branches that still exist, and the axes currently in the matrix. Nothing about the join requires reading the objects themselves, which is what makes it cheap enough to run monthly over a large corpus.

const reachable = new Set();
for (const branch of await liveBranches()) {
  for (const entry of await manifestAt(branch)) reachable.add(entry.hash);
}
const candidates = allManifestEntries.filter((e) => !reachable.has(e.hash));

The one subtlety is the grace period. A branch deleted this morning may be restored this afternoon, and a tag cut last week may still be the reference for a support release. Requiring a candidate to have been unreachable for a full period — thirty days is comfortable — before it is proposed removes almost all of that risk for almost no storage.

Unreference, then let storage collect

Pruning the manifest entry rather than the object Two pruning shapes are contrasted. Deleting objects directly removes the artifact permanently, breaks any manifest still naming that hash, is invisible in code review, and cannot be undone. Removing the manifest entry instead leaves the object in place, is a reviewable commit, is revertible, and lets the storage lifecycle rule collect the now-unreferenced object on its own schedule. A caption states the ordering that makes this safe: the manifest is the index, so an object with no manifest entry is already unreachable, and physical deletion becomes a storage concern rather than a correctness one. Unreference first; let storage collect later delete the objects permanent · breaks any manifest naming that hash · invisible in review a bisect that needed it now cannot run, and nothing records why remove the manifest entry a reviewable commit · revertible · the object stays until the lifecycle rule runs an unreferenced object is already unreachable, so deletion is a storage concern

Removing a manifest entry is a commit: it appears in review, it is attributable, and it reverts cleanly if the decision was wrong. Deleting an object is none of those things. Because the manifest is the index, an object nobody references is already unreachable in every practical sense, so the physical deletion can be left to a lifecycle rule that expires unreferenced objects after a further period.

{
  "Rules": [
    { "ID": "demote-superseded", "Filter": { "Prefix": "baselines/" },
      "Transitions": [{ "Days": 30, "StorageClass": "STANDARD_IA" }] },
    { "ID": "expire-unreferenced", "Filter": { "Tag": { "Key": "referenced", "Value": "false" } },
      "Expiration": { "Days": 120 } }
  ]
}

The two-stage arrangement means a mistaken unreference has four months to be noticed and reverted before anything becomes irrecoverable, and that the storage saving from demotion — which is most of it — arrives immediately regardless.

Propose, review, then apply

A pruning job that reports before it acts A scheduled pruning job runs in three passes. The first enumerates candidates by joining the manifest against the live branch list and the current matrix axes, producing a list with a reason per entry. The second publishes that list as a pull request description rather than acting on it, so a person sees what is about to become unreachable and why. The third applies the removals only after the pull request merges. A caption notes that the middle pass is what distinguishes a pruning job from a data-loss incident, and that its cost is one review per month. The middle pass is what makes it a job and not an incident 1 · enumerate manifest × branches × axes 2 · publish, do not act a pull request with reasons 3 · apply on merge remove the entries the cost is one review per month; the alternative is discovering the loss during an incident and a pruning job with no review pass will eventually run against a bad branch list

The pull request body is the artifact that matters. For each proposed removal it names the scenario, the axis cell, the reason it is unreachable, and the date it became so. A reviewer scanning that list is looking for one thing: an entry whose reason is wrong. In practice the wrong reasons cluster — a branch naming convention the job did not understand, a long-lived release branch that the forge reports oddly, an axis that was removed from the matrix declaration but is still being captured — and each one, once found, is a fix to the job rather than a one-off exception.

A pruning job without this pass will eventually run against a bad branch list. That is not a hypothetical risk: the branch list is the one input that comes from outside the repository, and it is the one most likely to be briefly wrong during a forge outage or a migration.

What to measure before pruning anything

Pruning is one of the few maintenance jobs whose value can be checked in advance, and doing so frequently changes the decision.

Total corpus size, and its growth rate. A corpus growing at a few gigabytes a year on object storage costs less than the engineering time to build a pruning job. One growing at a hundred gigabytes a quarter — usually a sign of a matrix that has been multiplied without an axis audit — has a different answer, and the better first move there is the axis collapse described in Cross-Browser Baseline Matrix rather than deleting the output.

The share that is superseded rather than current. If most of the corpus is current goldens, pruning cannot help; the size is the matrix. If most of it is superseded, the suite is re-blessing often, and it is worth asking why before reclaiming the storage — frequent re-blessing usually means an upstream determinism gap, and fixing that reduces the corpus permanently rather than periodically.

How often anyone actually reads an old baseline. Instrument the resolve path and count reads by age for a quarter. Teams commonly find that nothing older than six weeks has been read at all, which makes the retention decision straightforward, and occasionally find a long tail of reads during incident investigations, which makes it the opposite.

The measurement takes an afternoon and answers whether the rest of this page is worth implementing. It is entirely reasonable to conclude that it is not — a corpus that costs a few pounds a month and is never navigated by hand does not need a pruning job, and building one adds a scheduled process that can go wrong in exchange for a saving nobody would notice.

Verification

Confirm the procedure worked before wiring it into a blocking gate:

Troubleshooting

Symptom Likely cause Fix
A bisect cannot resolve a baseline from four months ago The expiry rule collected an object whose manifest entry had been removed earlier Lengthen the expiry window and confirm the demotion tier is doing the storage work rather than the expiry
The candidate list is enormous on the first run Every superseded baseline in the corpus is unreferenced by definition, and none has aged out yet Apply the grace period from the first run, and stage the initial pass over several months rather than all at once
A long-lived release branch’s baselines keep being proposed The branch list source excludes it — commonly because it is a protected branch reported through a different endpoint Fix the branch enumeration; an exception list papers over a job that will make the same mistake elsewhere

Frequently asked questions

How much storage does a baseline corpus actually use?

Less than teams expect, which is worth measuring before building any of this. A 1280 by 800 PNG capture is a few hundred kilobytes; a matrix of 200 scenarios across three engines and two device pixel ratios is roughly a gigabyte per full re-bless. Lossless WebP takes about a third off. Pruning is worth doing for hygiene and for keeping the corpus navigable, and it is frequently not worth doing for cost alone.

Should superseded baselines be kept forever for auditability?

The audit trail is the manifest history, which is in version control and costs nothing to keep indefinitely. The objects are only needed to look at an old frame, which is a bisect activity with a natural time horizon. Keeping the history forever and the pixels for a few months is the arrangement that matches how each is actually used.

What about baselines for an axis that was collapsed?

Treat the collapse as the event that makes them unreachable, and keep them through one grace period. A collapse is a claim about the content of a scene, and claims of that kind are occasionally wrong — the scene gains a vector label and the cells stop agreeing. Having the per-engine goldens still available makes reverting the collapse a manifest change rather than a re-capture.

Can the pruning job also find orphaned objects?

Yes, and it is worth the extra pass: objects in storage that no manifest in any branch’s history references at all. These accumulate from interrupted uploads and from experiments, and unlike superseded baselines they were never anybody’s reference. Report them separately from the reachability candidates, because the reasoning about them is different — an orphan has no history to lose.