The reframing tool does one job: take a 9:16 clip and outpaint it into 16:9, so a vertical ad can run on a widescreen placement without cropping the subject out. It's a young tool — its vendor integration dates back one month at the time of this audit — and it had reached the point where editors were running real, paid renders through it every day. That transition is where it stopped being a demo and started needing to survive redeploys, vendor flakiness, and its own money math being wrong.
Clean HTTP metrics don't mean nothing is failing. They mean nothing is failing at the layer you're measuring.
A render abandoned by a coincidence of timing
PR #1 traces a specific, maddening failure: a clip errors with "Lost connection to server... the render may have finished — check before re-running," which is a client-side give-up, not a real server error. The browser polls a status endpoint through a hub proxy in front of the tool, and while that hub redeploys — routinely 60 to 180 seconds of 502s on Railway — eight consecutive poll failures trip a client-side timeout and the UI marks the item as errored. The server was never told anything was wrong. The worker thread kept running. The job row survived. The editor just had no way to know that, so a perfectly good render looked abandoned because of unlucky timing against an unrelated deploy.
Clean metrics, invisible losses
PR #2 generalizes the fix, and its opening framing is the most useful sentence in the whole cluster: Railway's HTTP metrics were clean — 10.4k requests, zero 5xx, a flat 0.0% error rate. Nothing was failing at the web layer. Jobs were being lost, not erroring, and the only record of a failure was str(exc) in a dict that dies with the container the moment it restarts. Four distinct loss paths came out of the metrics and a production log — including the vendor rejecting a submission with a 429 when a shared account was busy — and the fix was to retry every vendor call and make failures diagnosable by writing them somewhere that outlives the process, instead of trusting a clean-looking dashboard to mean the system was healthy.
720p, billed as 1080p
PR #4 starts from an editor's finished render answering its own download link with Google's 401 page, an hour after the run completed with the file intact in Drive — and chasing that turned up something worse in the same batch of runs. Every run was supposed to use a specific native-1080p pipeline. Four independent measurements — wall-clock render time, bitrate, blur at 1fps sampled over 259 frames, and PSNR after a 1080→720→1080 round trip — separated the runs into two groups with no overlap between them:
| wall-clock | bitrate | blur (n=259) | PSNR round-trip | |
|---|---|---|---|---|
| native 1080p | 6.9 min median | 6.40–7.74 Mbps | 7.50 (sd 1.46) | 48.2–51.0 dB |
| downgraded to 720p | lower | lower | higher (softer) | lower (out of range) |
Seven of twenty-eight delivered segments had shipped as upscaled 720p while billed at native-1080p price — ten of thirty-seven segment rows were missing the field that records which upscale pass actually ran, while still reading status='complete'. The fix wasn't just correcting the price. It was serving the actual durable copy of a finished render instead of a transient one that could expire and start answering with a login page.
Two UI bugs nobody had reported, found by auditing the fix
PR #5 is the follow-up audit on #4, and it catches the kind of bug that survives specifically because it fails quietly in a direction nobody's watching. The segment progress strip branched its color logic on "done", "rendering", "processing", and "abandoned" — none of which is an actual value of the status field it was reading. The real "finished" value is complete; abandoned belongs to an entirely different enum. Only the failed branch ever matched anything real, so every complete, reused, or submitted segment rendered grey, as if it hadn't started — for the entire lifetime of the feature, with CSS that would have shown the correct color sitting right there, unused, the whole time. The fix pins the mapping end to end: a SQL CHECK constraint, an enum, and now two runtime guards, one in each direction, because the audit's own conclusion was that the second direction — catching an enum value with no matching UI branch — was the one that actually mattered.
The same missing-import bug, twice
PR #6 and PR #8 are the same failure mode a month apart, and the second time it happened is what made it worth fixing structurally rather than patching again. In #6, formatMin was called in two modules that never imported it — one inside a confirm() dialog string for the main "render N clips" button, meaning that button silently started zero renders, with no visible error, because the thrown ReferenceError pre-empted the dialog entirely. The per-row "run one clip" button worked fine, which is exactly why nobody noticed the batch button was dead. In #8, the same shape of bug hit APP_PREFIX: read in one module, imported in a sibling module it had been split out of, unreachable until a render actually succeeded and then thrown on every subsequent paint — so a row would sit on "Submitting..." for two real hours while the backend had already finished and failed nineteen minutes in. PR #9 is what actually closes the class of bug: a lint guard extended to flag any bare reference to an unimported binding, not just function calls — and the guard itself had a gap, since testing a name inside an if (APP_PREFIX) condition had been silently counted as a valid "use" that proved the name was bound, hiding 91 such cases across the codebase from the very check meant to catch them.
An adopted row that couldn't actually retry
PR #10 is a sharp one: a row restored from a prior partial run — an "adopted" row — carries a placeholder file object with size: 0, because the real bytes live on the server and the browser tab looking at it never had them. That fact had been recorded on the row since PR #3. No code had ever read it. So clicking "Retry anyway" on an adopted row ran the exact same fallback path as a fresh upload — FormData.append("video", placeholder) — which JavaScript happily stringifies into the literal text "[object Object]" as a form field, sent to a server that (correctly) rejected it as garbage, discarding a render that should have simply been re-run server-side from data it already had.
Two deprecated vendor models and a fallback that made things worse
PR #11 through #15 form the last arc, and it starts with a question someone asked directly: is the model outdated? PR #12 confirms it in the vendor's own words — both models the tool was using are explicitly deprecated, and the integration dated to the tool's very first commit, a month before the replacement model even shipped, never revisited since. But before reaching for the obvious "just upgrade," PR #11 measured what was actually happening: across 13 days and 64 submissions, the fast tier of the old model had a 0% failure-to-start rate; the tier the system fell back to when something went wrong had a 25% failure-to-start rate and was reached exclusively through the exact failure path meant to recover from trouble. The recovery path was less reliable than what it was recovering from — a fallback that can make an outage worse than doing nothing.
Moving to the current model (PR #12) opened a second, subtler problem: the two vendor integration paths — "brokers," in the tool's own language — disagreed with each other's documentation about a basic parameter. One broker's docs said an omitted duration field defaults to matching the source; the other's said it defaults to a flat 5 seconds. PR #13 fixed the code to match what each vendor's docs claimed. PR #14 then paid for three real generations to check the docs against reality, and found two of the three documented behaviors were simply wrong — the field made no observable difference to either broker's output. PR #15 finally exercised the previously-never-successful integration path end to end with a real purchase, confirming the correct behavior directly rather than trusting anyone's documentation, including the vendor's own.
The pattern: measure the money, not just the uptime
Nothing in this cluster failed loudly. A render abandoned by a proxy redeploy still looked like a normal error. A dashboard reporting 0% error rate was telling the truth about the layer it measured and nothing about the layer that mattered. Wrong-resolution output still played back fine in a preview. A fallback path can pass code review and still be worse than not falling back at all. The throughline across all fifteen PRs is the same: don't trust an instrument that's clean by construction — measure the actual bytes delivered, the actual price charged, and the actual vendor behavior under a real paid call, because that's the only place these bugs were ever visible.