/runs had no queryable store. The only durable copy of a run's state was one job-state.json file per run, sitting in that run's own Google Drive folder — so rendering a table of thirteen fields meant enumerating every Drive folder and downloading every run's entire job document to read them. Measured: a mean of 65 KB and a max of 445 KB per run, for a table that showed thirteen columns. The in-memory cache that was supposed to make this bearable was module-level, so the first load after every deploy returned an empty list. A sibling tool's own schema file had already named the smell: "read live state from a 5-minute-stale Drive folder scan."
Four of this round's defects were mine, and every one was found by measuring production rather than by reading the diff.
That's not a stray line — it's the author's own summary of the nine-PR round that closed this migration out. It's worth sitting with, because it sets the tone for everything below: replacing a Drive scan with a real index sounds like a clean infrastructure swap, and it produced instead a small, honest cluster of self-found production bugs, several of them shipped by the same person who then caught them.
A queryable index, kept deliberately fail-open
PR #890 built a Supabase-backed run index as a projection, and made one important choice explicit: the old Drive-scan path didn't get deleted yet. It stayed as a fallback if Supabase failed, and — just as useful — as a free oracle to diff the new projection against. Two systems both answering "what runs exist" isn't waste when one of them exists specifically to catch the other being wrong.
That oracle earned its keep. Run twice over the same production data, the comparison came back $0 both times: 293 of 293 records identical, then 290 of 290, with job-ID sets matching exactly and 14 of 16 fields byte-identical on every row. The one legitimate difference was a projector-written row that was a few hours fresher than its frozen Drive snapshot — which is the index being right, not the index disagreeing. Only once that verification held up did PR #928 retire the Drive scan and make the index the single source of truth, closing the issue that had opened the whole migration.
A cache that became a write's merge base by accident
The migration's first real data-loss bug arrived immediately, in PR #892, and it's a sharp lesson in how a read path and a write path can silently start sharing state that was never designed to be shared. Every catalog writer in the pipeline is a read-modify-write: read the whole index, merge one record into it, upload the merged result back over the Drive copy. PR #890 had made the public listProfiles()/listRips() functions prefer the new Supabase mirror — and those were exactly the functions the writers used as their merge base. The mirror write was deliberately fail-open, so it could silently fail while the Drive upload succeeded, leaving the mirror stale. The next writer would then read that stale mirror, merge its own change in, and upload the result over Drive — permanently erasing whatever had changed in between. A cache that's allowed to be wrong is fine for reads. It is never allowed to be the base of a write.
The same shape reappeared for a second catalog a few days later. PR #910 found that setLabel — the writer behind display-name renames — re-read the current state before writing, which is the correct instinct against a stale cache. But its re-read path, ensureLabelsFresh plus currentLabels(), could not report failure: it failed open on a Drive error, and on a cold container with no existing cache, its fallback was an empty object. A cold container plus one failed Drive request was enough to upload an effectively-empty map over every rename anyone had ever made. PR #924 closed it by mirroring the display-name index into the same Supabase system as the others — the fifth and last of five app-written Drive-JSON indexes, and the only one still reading Drive on a cold start.
A run's real spend, not just its final-video cost
PR #893 is the kind of bug that's invisible until someone reads a real number. total_cost_usd summed models[].costUsd — the per-model cost of a finished video. A run that paused at a review gate, or failed before producing a final video, reported $0 spent no matter how much detection and planning it had actually paid for. This was found by watching the very first row the new projector wrote in production: a run parked mid-pipeline, genuinely billed, reading as free. The "spend by week" query shipped in the same schema summed exactly that broken column.
The deeper fix landed in PR #930, and its own framing is the cleanest summary: "the delta was in hand and thrown away." The run's live ledger — job.costSoFarUsd plus a list of cost notes — lived only in-memory, persisted solely inside the Drive snapshot. Giving the run total a durable home (PR #893) wasn't enough, because a total can't answer "which specific charge was this for" — and that's exactly the question both of this round's documented money bugs turned on. The function that recorded every charge, addCost, already computed the exact dollar delta for each one. It just wrote that number to a log line and discarded it instead of persisting it anywhere durable.
// before: the delta existed for one log line and nowhere else
function addCost(job: Job, deltaUsd: number, note: string) {
job.costSoFarUsd += deltaUsd;
console.log(`[cost] +$${deltaUsd.toFixed(2)} — ${note}`);
// deltaUsd is gone the moment this function returns
}
// after: the same number, persisted per-charge
function addCost(job: Job, deltaUsd: number, note: string) {
job.costSoFarUsd += deltaUsd;
ledger.record({ runId: job.id, deltaUsd, note, at: Date.now() });
}PR #907 used that same underlying accounting to answer a question that had never had a real number attached to it: when an operator cancels a wedged run, how much money are they walking away from? Before this, "it will stop and be marked failed" was the entire message — no figure, no way to distinguish killing a forty-cent run from killing a fourteen-dollar one. A dedicated module now derives three figures — spent, committed, and not-yet-started — specifically so the number that actually matters for a cancel decision has somewhere to be shown.
The age-out bug that emptied the dashboard's most important group
This is the round's most instructive regression, because it's a bug introduced by a refactor that was itself a fix. PR #925 moved the stalled-run age-out logic from a client-side module into the server-side run-index reader — a legitimate move, made its own PR specifically because the team's own rule says a deletion must never share a PR with a behavior change. But the move carried the threshold across and quietly dropped the ordering of the checks — and the ordering turned out to be the whole rule.
The display status of a run is decided in three sequential steps: terminal state, then review-gate state, then freshness/staleness. Ageing a row out on the server rewrote its status to a terminal value directly, which meant the review-gate check downstream could never fire for it. A run sitting at a review gate isn't stalled — it's waiting for a human, has no running process that could have died, and its idle time carries no information at all. PR #927 caught the consequence in production: the age-out was emptying the dashboard's "needs you" group, silently hiding exactly the runs an operator most needed to see. The fix wasn't reverting the move — it was restoring the order the original code had gotten right by accident.
Deleting the code that used to be true
Alongside the index migration, a separate but related discipline ran through the same weeks: finding and removing code nobody calls anymore, using tools rather than instinct. PR #851 used the repo's own call-graph script to find two genuinely dead functions among thirty-four flagged candidates — most of the rest were framework callbacks the tool correctly knows to warn about rather than delete. PR #856 found three scratch files sitting at the repo root, invisible to every guard because the guard's glob only matched .ts/.tsx — the fourth time in this effort that a coverage gap, not a threshold, was the actual defect.
PR #906 is the capstone: two commits, the first clearing 1,747 dead import bindings left behind by an earlier file-splitting migration (every extracted file had kept importing what its pre-split self used, and nothing in the existing lint config could see the leftovers), the second turning on no-unused-vars so the count can't silently grow back. Fixing the tooling itself surfaced real defects: a file-wide bracket cleanup regex had turned insertShape: shape,\n} into a syntactically valid but semantically broken line across 162 files, and three separate source-scanning tools had been quietly missing files because of it. The sweep found four real bugs this way — not by someone going looking for bugs, but by making 1,897 pieces of dead weight visible enough that the four live wires tangled up in them couldn't hide anymore.
Not every "dead code" candidate survives contact with measurement, and that's worth stating plainly rather than glossing over. PR #973 audited eight items flagged as unread and found five were load-bearing — one route had four live callers serving labels to three different surfaces, one field appeared in eight separate schema assertions governing what a forked run inherits. The count of what's actually safe to delete keeps shrinking under real inspection, which is the entire point of measuring before cutting rather than after.
A security deletion, not a hardening
PR #921 is a smaller finding with an outsized blast radius if it had gone unnoticed longer: a legacy image proxy sat on a route excluded from the hub's auth gate, ostensibly so an image-editing model could pull frames by URL. In practice, it fetched an arbitrary URL out of job state and streamed the response body back — an unauthenticated request relay running from the service's own egress IP, returning someone else's content under this origin. Measured before deleting rather than assumed: of 86 archived job-state documents, exactly two carried the field this route needed, and every URL in both pointed at the service's own host. The fix was deletion, not hardening, because by the time anyone looked, the branch was already dead — which didn't make it any less of a live vulnerability while it existed.
The pattern: infrastructure migrations create their own bug class
Nothing in this cluster is a hard algorithm. A cache became a write's merge base because two independently-reasonable pieces of code started sharing state neither was designed to share. An age-out check regressed because a refactor preserved the threshold and silently dropped the order it ran in. A cost total was wrong because "spent" and "billed for a finished video" look identical until a run stops partway through. Every one of these is the specific bug class that infrastructure migrations create: not the new system being wrong, but the seam between the new system and everything that still assumes the old one — caches, readers, orderings, assumptions about what a field means — being wrong in ways that only show up once real production data runs through it. The fix, consistently, was the same discipline repeated four times: measure the live behavior, not the diff.