"Approve" is the button that turns a plan into paid clips. On the main video-generation service, that button sits behind a storyboard gate: every establishing frame gets checked against the reference ad it's swiping — same instruments in frame, same hands occupied or empty, same product visible where the reference shows it. The idea is simple. The implementation spent a month finding out that a check which is itself unverified is just a second place for the bug to hide.
A gate that can be wrong is not a gate. It's an opinion with a veto.
A gate that compared the reference to itself
PR #773 is the cleanest example of the whole month's theme. A check called cutaway-ad-as-talking-head blocked auto-advance with the message "our plan covers 38% of it with cutaways" — a number that sounded like a real measurement of the plan. It wasn't. gate.ts supplied referencePersonlessShare(referenceShots) as the stand-in for "our share," so both sides of the comparison were measuring the reference, with two different instruments, and the check fired on their disagreement with each other rather than on any actual gap between plan and reference. The plan was never consulted.
PR #774 is the same failure mode from the opposite direction: the board gate refused to approve a plan it had already paid fourteen frames for ($5.87), because a critic looked at a cutaway render of the pipeline's own device and read it as a different device. The plan's motion description for that panel explicitly called for a translucent variant — the critic was correct that panel 4 looked different from panel 8, and wrong that "different" meant "wrong." A board that "does not agree with itself" is a real signal worth having. It just needs to be checked against what was actually asked for, not against an average of its own panels.
A reroll loop that could never close
PR #780 found a genuinely nasty state bug: the board gate's designed remedy for a bad panel is a cheap reroll (~$0.10 via regenerate with ugc-dyn-frame) instead of a full re-render (~$1.15). The reroll invalidated the old checkpoint and rendered a new frame — and then never re-graded it. approve kept reading the stale failed[] array from before the reroll, so once a panel triggered a failure, the loop had no path back to a clean state. The fix isn't clever; it's just closing the loop the first version left open, which is exactly the kind of bug that only shows up when someone actually exercises the recovery path instead of only testing the happy one.
PR #785 caught what shipped right behind that fix: after a reroll re-graded correctly for the first time, the board wrote storyboardReport: checked: true, panels: 2, failed: [] on a run with fourteen scenes and fourteen rendered frames. A clean pass over 2 of 14 panels read as a pass on the whole board — on the exact gate guarding roughly $16 of clips per run. Probing all fourteen frames directly showed every one of them was still fetchable and gradable; nothing was actually lost, the report was just wrong about how much of the board it had covered. A short sheet reading as a full pass is worse than a gate that's too strict, because it looks identical to success from the outside.
Messages that reported the question, not the answer
PR #782 is a smaller fix with an outsized effect on trust in the system: seven gate messages led with the fixed, code-authored criterion instead of the model's actual finding. The real answer was there — thirty to eighty words later, in a trailing parenthetical the operator had to dig past boilerplate to reach, on the exact 409 response a human reads before deciding whether to override a block. When the thing you show first is the question rather than the answer, every gate reads as more opaque than it is, independent of whether the underlying check is right.
Three PRs to find the real ceiling
The token-limit saga across #796, #803, and #805 is worth walking through in full because it's a clean example of iterating toward a root cause instead of stopping at the first plausible fix. A two-hander re-plan (two people in one scene, doubling the planning complexity) truncated at a 16,000-token ceiling. PR #796 raised it to 32,000 — reasonable on its face. It broke every plan call in production within minutes: the Anthropic SDK refuses a non-streaming request whose max_tokens implies a response that could run over ten minutes, and the run function called messages.create, not messages.stream. PR #803 reverted the raise as an emergency fix, with an honest note that its own original reasoning had been wrong. PR #805 found the actual root cause at the SDK source rather than inferring it from docs, and fixed the transport instead of the symptom: stream every call, so the real ceiling becomes whatever the model itself supports, not an arbitrary number picked to dodge a client-side timeout heuristic. The two-hander plan that #796 was trying to unblock finally worked — through the fix that took three PRs to actually name the constraint correctly.
Facts that died before the vendor call
PR #781 traces a contradiction all the way to its source: a composite closing shot — presenter, brand mascot, and several product units grouped on a table, mirroring the reference's own end card — was fully wired in the data model and still rendered wrong. The scene's plan data was completely correct: mascotEndCard: true, the reference board correctly typed as an end-card at that position, the mascot correctly anchored to bookend the reference's own opening shot. The frame that actually rendered was a macro of one closed case, alone. The facts were right at every stage of the pipeline except the last one — they simply never survived into the prompt actually sent to the image model.
PR #798 found the same family of bug from a rendering angle: a library b-roll span painted over 41.9 to 45.5 seconds of a delivered master's closing composite shot, so the composite showed for about 1.2 of its intended 5 seconds. The root cause: end-card is a member of CUTAWAY_SHOT_TYPES, so whenever the reference's own last shot happens to be typed end-card, the ordinary b-roll span selector treats it as fair game for a cutaway candidate — nothing in that selector knew a composite closing shot needed protection from the exact category it was classified under.
A rule that kept overruling its own plan
PR #792 documents a recurring bug in the staging logic: an "empty hands" default rule kept overriding scenes where the plan explicitly described hands as occupied — this time with the presenter's motion reading "slides the closed case into his cardigan pocket and pats it flat," and the rendered frame prompt asserting "BOTH HANDS ARE COMPLETELY EMPTY." The PR notes this is the third recurrence with new verbs, after twelve prior beats across six runs — which is exactly the kind of pattern that signals a detector matching against a fixed vocabulary rather than the actual semantic claim, and each new verb choice in the script slips past it again.
What "measure, don't guess" restraint looks like
PR #800 is worth including for what it deliberately doesn't do. Comparing the reference and the pipeline's own output, the reference consistently builds toward the product touching the body — hands massaging cream into skin, an overlay glowing through contact — while the pipeline's beats never show the device making contact at all. The natural fix is to auto-generate a contact beat. The PR ships measurement only: a contact table across all seven reference/output pairs, agreeing exactly on the gap, with an explicit note that auto-generating the missing beat is a separate, riskier decision than measuring that it's missing. Not every finding needs to become a feature in the same PR that discovers it.
The pattern: verify the verifier
Every fix in this cluster is one layer removed from the bug you'd expect. The storyboard gate isn't "buggy" in the sense of producing wrong grades on correct inputs most of the time — it's buggy in the sense that the machinery grading the ad needed exactly as much scrutiny as the machinery generating it, and for a while got less. A gate that compares the reference to itself, a critic that can't distinguish its own product's variants, a reroll that never re-grades, a message that shows the question instead of the answer — none of these are algorithmically hard. They're the direct consequence of a verification layer being trusted the moment it started reporting failures, instead of being measured with the same discipline applied to the thing it verifies.