Performance marketers do not need another full ad from scratch. They need controlled variation — keep the hook that converts, swap the middle section that fatigued, leave the CTA intact. Variant Multiplier is the tool I built on the AI ad platform to do exactly that: take a winning ad, isolate one section, rewrite the lines, regenerate B-roll, and re-stitch the result into a fresh variant that still feels like the same ad.
The product shipped across 23 merged PRs, from hardened core foundation through a six-workstream studio build. This post covers the concept, the studio architecture, and the technical patterns — waveform section picking, word-level transcript selection, grade-match splicing, end-frame interpolation, batch queue, and live cost estimation — that made section-swapping production-ready.
Full regeneration is expensive and risky. Section swap is surgical: one cut, one rewrite, one re-stitch — the rest of the winning ad stays untouched.
The product concept
A typical winning UGC ad has three structural sections: hook (0–3s), body (product proof + testimonial), and close (offer + CTA). When the body fatigues in Meta's auction, editors want a new body without touching the hook that earned the click or the close that drives conversion.
Variant Multiplier's pipeline:
- Upload — editor drops the winning ad (Google Drive or local)
- Select section — scrub waveform, pick start/end via silence peaks or word-level transcript
- Rewrite — LLM rewrites lines for the selected section only
- Regenerate — B-roll and VO for the swapped section through video + TTS APIs
- Re-stitch — grade-match + end-frame interpolation at splice points
- Compare — side-by-side A/B of original vs variant
| Step | Editor action | System responsibility | Typical cost |
|---|---|---|---|
| Section pick | Scrub + drag handles | Waveform, silence detection, transcript alignment | $0 |
| Rewrite | Review / edit lines | LLM section rewrite with product context | ~$0.02 |
| B-roll regen | Approve look-match frame | Nano Banana Pro look-match + Veo/fal generation | $0.40–$1.80 |
| VO regen | Pre-spend QA listen | ElevenLabs clone + FFmpeg sync | ~$0.08 |
| Stitch | Download or push to Drive | Grade-match, end-frame, concat | $0 |
Foundation: hardened core and roadmap
The first PRs rebuilt Variant Multiplier on a hardened core shared with sibling video-editing tools on the platform. That meant Docker build fixes, consistent error boundaries, structured logging, and a shared Next.js app shell. Roadmap docs defined six studio workstreams before any UI shipped — a deliberate choice to avoid building features that could not compose.
// Core run manifest — every section-swap run carries this shape
interface SectionSwapRun {
id: string;
name: string; // collision-proof, set at upload
sourceVideoUrl: string;
section: {
startMs: number;
endMs: number;
transcriptSpan?: WordSpan[]; // word-level selection
mode: 'rewrite-broll' | 'rewrite-vo' | 'full-regen';
};
rewrite: SectionRewrite;
estimate: CostEstimate; // live preview before spend
status: RunStatus;
outputs: { variantA?: string; variantB?: string };
}Naming at upload (feat/run-naming-rename) solved a storage collision problem early. Editors name runs when they upload; renames propagate to Google Drive mirrors; collision-proof storage keys prevent overwrites when two editors swap sections on the same source ad simultaneously.
The six-workstream studio build
The studio UI shipped as six parallel workstreams, each a focused PR cluster:
| Workstream | PR | Deliverable |
|---|---|---|
| WS1 — Backend | feat/studio-ws1-backend |
Silence/peak API, cost estimator, config/estimate endpoint |
| WS2 — Transcript | feat/studio-ws2-transcript |
Transcript selection + waveform/silence timeline |
| WS3 — Modes + cost | feat/studio-ws3-modes-cost |
Per-section modes + live cost preview |
| WS4 — Section picker UX | feat/ux-section-picker |
Watch-and-scrub picker, no-auto-spend, progress/cancel |
| WS5–6 — Compare + onboarding | feat/studio-ws5-6 |
Variant A/B compare + first-run onboarding |
| Cross-cutting | feat/ui-consistency |
Design system alignment with sibling VE tools |
WS1 exposed backend primitives the UI could compose without tight coupling. Silence detection and peak extraction ran server-side on uploaded audio; the cost estimator consumed section duration, selected mode, and engine config to return a dollar estimate before any API call fired.
// POST /api/runs/:id/config/estimate
async function estimateSectionSwapCost(
section: SectionBounds,
mode: SectionMode,
engines: EngineConfig,
): Promise<CostEstimate> {
const durationSec = (section.endMs - section.startMs) / 1000;
const brollCost = mode.includesBroll
? engines.video.pricePerSecond * durationSec
: 0;
const voCost = mode.includesVo
? engines.tts.pricePerCharacter * estimateCharCount(section)
: 0;
const rewriteCost = engines.llm.pricePer1kTokens * REWRITE_TOKEN_BUDGET;
return {
totalUsd: brollCost + voCost + rewriteCost,
breakdown: { broll: brollCost, vo: voCost, rewrite: rewriteCost },
engines: resolveEngines(mode, engines),
};
}Waveform and silence timeline for section selection
Editors think in beats, not milliseconds. The section picker overlays a waveform with silence gaps highlighted — natural cut points where the presenter pauses between sentences. Peaks mark emphasis; valleys mark breath gaps.
interface SilenceSegment {
startMs: number;
endMs: number;
confidence: number;
}
function snapHandlesToSilence(
startMs: number,
endMs: number,
silences: SilenceSegment[],
snapThresholdMs = 120,
): SectionBounds {
const snapStart = silences.find(s =>
Math.abs(s.startMs - startMs) < snapThresholdMs
);
const snapEnd = silences.find(s =>
Math.abs(s.endMs - endMs) < snapThresholdMs
);
return {
startMs: snapStart?.startMs ?? startMs,
endMs: snapEnd?.endMs ?? endMs,
};
}The watch-and-scrub UX (feat/ux-section-picker) plays video synchronized to handle drag. Editors see exactly what they are cutting. No-auto-spend means scrubbing and selecting never triggers generation — spend only happens on explicit "Run" after cost preview confirmation.
Word-level transcript selection
Silence snapping gets editors close, but copywriters think in words. Word-level (karaoke-style) transcript selection lets editors click individual words to set section boundaries — "from 'actually' through 'results'" rather than "from 12.4s through 18.7s."
interface WordSpan {
word: string;
startMs: number;
endMs: number;
index: number;
}
function selectionFromWordRange(
words: WordSpan[],
startIndex: number,
endIndex: number,
): SectionBounds {
return {
startMs: words[startIndex].startMs,
endMs: words[endIndex].endMs,
transcriptSpan: words.slice(startIndex, endIndex + 1),
};
}Word spans feed the rewrite step with exact transcript context. The LLM receives the selected words plus surrounding lines for continuity, not a raw timestamp range.
Look-match with Nano Banana Pro
Regenerated B-roll must match the source ad's visual language — lighting, palette, setting. The feat/gemini-image PR added a Nano Banana Pro (Gemini) look-match backend: extract a reference frame from the section boundary, generate a style-locked seed frame, and pass it to the video engine as the first-frame anchor.
This prevents the classic section-swap failure mode: regenerated middle looks like a different shoot entirely while hook and close remain warm UGC.
Grade-match and end-frame interpolation
Even with look-match, splice points show colour discontinuity. Two PRs addressed this:
- Grade-match (
feat/grade-match) — deterministic colour transfer from source boundary frame to generated section's first/last frame - End-frame interpolation (
feat/end-frame) — generate a transition frame that smoothly bridges the cut-back to original footage
I cover the colour science in depth in Seam-Free Video Splicing. The short version: deterministic histogram matching beats neural colour grading at production scale because it is reproducible, fast, and does not hallucinate.
Pre-spend voiceover QA
VO regeneration is cheap relative to video, but bad VO ruins an otherwise good variant. feat/vo-qa added a listen-before-spend step: generate VO preview, surface waveform + transcript alignment, block stitch until editor approves or regenerates.
Batch queue and recent runs
Editors rarely swap one section. They batch: new body for variant A, new hook-adjacent transition for variant B, new close for variant C. feat/batch-queue runs section swaps back-to-back with shared source upload and independent cost estimates per job.
feat/recent-runs surfaces live runs unioned with Google Drive mirrors — if a run completed while the editor was in another tab, it appears in recent history without a refresh. Drive durability PRs (feat/drive-durability-and-api-fixes, streaming fixes) ensured large video files stream without crashing the Node process on memory pressure.
Reliability fixes
| PR | Problem | Fix |
|---|---|---|
fix/drive-media-streaming |
Full file buffered in memory | Chunked streaming from Drive API |
fix/drive-stream-crashsafe |
Process crash on stream abort | Crash-safe stream teardown + retry |
fix/fal-veo-transient-422 |
fal Veo soft-reject on busy periods | Exponential backoff retry on 422 |
fix/e2e-variant-assert |
Stitch regressions undetected | E2E assertion on variant output duration |
What I'd do differently
I would ship word-level selection in WS2 instead of as a follow-up PR. Editors who tried silence snapping first developed a mental model that word selection partially contradicted — click-to-word felt like a different tool until we unified the timeline UI.
I would also embed cost estimate in the section picker handles — showing estimated spend as the editor drags section boundaries longer, not only on the review screen before run.
The pattern: surgical variation over full regeneration
Variant Multiplier encodes a product principle: preserve what wins, vary what fatigues. The engineering follows — section-bounded rewrites, look-matched regeneration, deterministic splice repair, and fail-before-spend guards at every API boundary. Twenty-three PRs sounds like a lot for one tool; most of them are the unglamorous infrastructure that makes a three-click editor workflow possible.