Presenter Generation renders an AI avatar delivering a script — cartoon or realistic, lip-synced, aspect-ratio aware. A test pass from a teammate flagged two problems from two real runs. What actually shipped across the next eight PRs was closer to a forensic audit: read the job-state and log files each failed run had left on disk, trust nothing that wasn't directly measurable from them, and follow each finding to whatever it actually was rather than the first plausible explanation.
"The audio came back different" proves nothing against a non-deterministic model. Measure the transcript, not the vibe.
An aspect-ratio bug that reproduced on every run
PR #20 found two problems that turned out to be different in kind. The first — every run set to 16:9 rendering 9:16 anyway — was a straightforward code defect, reproducible on every single run, confirmed directly from the job-state JSON: the settings recorded 16:9, the delivered file was 9:16. The second was a cartoon lip-sync desync, and it looked similar from the outside but wasn't: it was dominated by two production environment flags, with a real but smaller code gap underneath. Treating both symptoms as one bug would have meant chasing a code fix for something that was actually a misconfigured flag, or shipping a flag change that left the real underlying gap in place.
A flaky call that silently disarmed a spend gate
PR #21 investigated a continuity-canon call that failed with a bare, fieldless 400 error — "invalid request data," no further detail — after which the run continued without a canon reference at all. Rather than accept the first plausible cause, the investigation ruled three out explicitly: not the response schema, because another run's canon call succeeded on the identical schema minutes earlier; not image size, because the payload was well under the vendor's documented per-image limit. The actual finding was narrower and more concerning than any of those: the failure silently disarmed a downstream lip-sync spend gate, meaning a transient, unexplained 400 on one call could remove a cost control from a run with nobody deciding that should happen.
A flag that covered two of three warp sites
PR #22 made no behavior change at all — a comment and a log-level adjustment — and it's still worth including because of what it documents. A run had shipped a desynced cartoon because a "never warp the picture" flag was off in the production environment, letting the picture speed up 1.3x before a lip-sync model was asked to paint a mouth onto frames that no longer matched their original timing. Reconciling environment variables afterward confirmed the override had already been fixed at the config layer — there was nothing left to fix in code. But the flag's own name promised more than it delivered: it covered two of the three call sites that could warp a picture, silently, and that gap needed to be on record rather than rediscovered the next time someone reasonably assumed "never warp" meant never, anywhere.
A lexicon attached to a synthesis nobody hears, twice
PR #23 found that a pronunciation dictionary was firing correctly — the logs showed "dictionary ready" and listed its rules — right before the run shipped a mispronounced word anyway. The dictionary was attached to a voice synthesis call that fed a different downstream path than the one actually bound to the delivered audio; being technically correct and being heard are not the same claim, and only checking the log line for "dictionary ready" would have missed that the correct synthesis and the shipped audio had quietly diverged. The same PR found the pose lock — meant to keep a character's pose stable across a run — had been implemented as this repo's own addition rather than inherited correctly from elsewhere, worth flagging because it meant future pose bugs needed to be diagnosed against this codebase specifically, not assumed to behave like a sibling tool's version.
A green checkmark that was wrong twice over
PR #24 is the sharpest self-correction in the batch. A probe PR #23 had shipped but never actually run was finally executed, and it passed — on a transcript that read "I am your cochlear, uh" against a prompt that intended "cochlea." The assertion was a plain substring match, and "cochlear".includes("cochlea") is true, so the check reported success on a word that was, in fact, wrong. Worse, the same loose match would have passed on "cochlear implant" too. The fix adds a word-boundary match plus an explicit check that a deliberately awkward phonetic respelling — meant to be read as one word — hadn't instead been read by the voice model as several separate words strung together, which is its own distinct failure mode a simple substring test can't see at all.
A fix that shipped on and made things worse
PR #25 is an honest admission built into its own PR description: of the four fixes in the previous round, three were confirmed working end to end and one wasn't — and that one had shipped turned on by default. An "adopt this frame and sharpen it" step, applied to a generative image edit, cannot actually leave the frame unchanged the way a deterministic sharpen filter would; the model regenerates the whole frame from the instruction, and despite the prompt explicitly forbidding re-posing, re-framing, recoloring, and resizing, the character came back larger, repositioned, and with its details altered anyway. The fix wasn't a better prompt — it was defaulting the step off, because a generative model told "change nothing but do this one visible thing" reliably fails to hold the "nothing" part.
Paying to render pixels and then binning them
PR #26 found that a widely-repeated technical constraint had quietly stopped being true. The frame pipeline had been built around "the image model has no aspect-ratio parameter, only three fixed sizes, none of which is 16:9" — accurate for one version of that model, and no longer accurate for the version actually in use, which takes an arbitrary width-by-height size field that dictates output shape directly. The pipeline had spent months rendering a frame at the wrong shape and cropping or padding it afterward — paying full generation cost for pixels it would then discard — when the fix was to ask for the correct shape from the vendor in the first place and pay for exactly the pixels that were needed.
Decisions logged to a file a redeploy wipes
PR #27 closes the loop on the whole investigation with an infrastructure gap: two per-clip decisions — which words were respelled for the voice model, and what delivery direction, if any, was given — were logged as plain application-log lines. The application log lives in the container's ephemeral filesystem. A routine Railway redeploy wipes it, meaning the exact record an after-the-fact investigation like this entire cluster depended on could vanish before anyone thought to look for it, purely from unrelated deploy timing. The fix persists those two decisions somewhere a redeploy can't reach, closing the same gap that had made every earlier PR in this cluster harder than it needed to be: several of these bugs took multiple rounds to fully diagnose specifically because the evidence describing what a run actually did wasn't guaranteed to still exist by the time anyone went looking.
The pattern: read the artifact, not the assumption
Every fix in this cluster traces back to checking a specific, concrete artifact — a job-state file, a log line, a rendered frame, a transcript — instead of trusting the plausible story about what the code should have done. The green checkmark in #24 is the cleanest illustration: a test existed, ran, and passed, and was still wrong, because nobody had checked what the assertion actually matched against. The fix each time wasn't cleverness. It was reading the real output before believing the report about it.