In short-form video advertising, the first three seconds are not a warm-up. They are the product. That makes leading silence unusually expensive. A 300ms pause is normal in a long voiceover; inside a 3-second hook it consumes ten percent of the creative before the first word lands. In Hook Machine, generated hooks were technically valid, but some opened with a dead-air gap from the voice model before speech began.
The first implementation only warned about the problem. That was useful for debugging and useless for production. Editors do not want a report that says the hook starts late; they want the rendered clip to start cleanly. PR #55 (feat/trim-vo-lead-in) changed the hook pipeline from "detect and warn" to "detect, trim, retime, and render." PR #58 (fix/audio-tag-blind-spot) caught the neighboring failure: an inline audio element made the hook look fine in the browser while the captured frames still burned a late-starting audio control into the output.
In 3-second ad openers, 300ms of dead air is 10% of total runtime. A hook renderer should fix that automatically, not ask an editor to notice it.
The late-starting hook problem
Generated voiceover has natural variance. The same text can produce different leading silence depending on voice, model settings, normalization, and punctuation. Static trims fail because they either leave silence in some outputs or cut the start of the first phoneme in others. The render system needed to measure the actual audio artifact before composing the final hook.
| Failure | User-visible result | Why warnings were insufficient |
|---|---|---|
| 200-800ms TTS lead-in | Hook feels slow before the first word | The editor already paid for a candidate that needs manual trimming |
| Audio-only trim | Voice starts on time, avatar mouth starts late | The output looks desynchronized even when waveform timing is fixed |
| DOM audio capture | Player chrome or hidden audio state appears in frames | The issue is invisible until the final encoded video is reviewed |
This is the same class of problem as Viral Feel Parity: the model output can be "correct" while the final creative feels wrong. The fix has to live in the renderer, where audio, frames, captions, and final encoding meet.
Detecting silence thresholds programmatically
The core detector runs FFmpeg's silencedetect filter against the rendered voiceover, parses the first silence_end, and clamps it through guardrails. The clamp matters because noise floors vary. A tiny breath at the start should not shift the entire hook; a true half-second lead-in should.
import { execa } from 'execa';
type LeadInTrim = {
trimMs: number;
reason: 'silence_detected' | 'below_threshold' | 'no_audio';
};
export async function detectVoiceLeadIn(
audioPath: string,
noiseDb = -45,
): Promise<LeadInTrim> {
const { stderr } = await execa('ffmpeg', [
'-i', audioPath,
'-af', `silencedetect=noise=${noiseDb}dB:d=0.05`,
'-f', 'null',
'-',
], { reject: false });
const match = stderr.match(/silence_end:\s*([\d.]+)/);
if (!match) return { trimMs: 0, reason: 'below_threshold' };
const trimMs = Math.round(Number(match[1]) * 1000);
return {
trimMs: trimMs >= 80 ? trimMs : 0,
reason: trimMs >= 80 ? 'silence_detected' : 'below_threshold',
};
}The threshold is deliberately conservative. Trimming a clean hook by 20ms is more dangerous than leaving 20ms of room tone. The render only shifts when silence is large enough for a viewer to feel it.
Retiming audio, frames, and captions together
Once the lead-in is measured, the renderer cannot simply cut the audio file. The hook is a composed artifact: avatar frames, caption word timings, audio bed, and final mux all need the same timeline origin. PR #55 treated trim as a manifest transformation rather than an FFmpeg one-off.
type HookTimeline = {
voicePath: string;
videoFramesDir: string;
words: Array<{ text: string; startMs: number; endMs: number }>;
trimMs: number;
};
function applyLeadInTrim(timeline: HookTimeline): HookTimeline {
if (!timeline.trimMs) return timeline;
return {
...timeline,
words: timeline.words.map(word => ({
...word,
startMs: Math.max(0, word.startMs - timeline.trimMs),
endMs: Math.max(0, word.endMs - timeline.trimMs),
})),
};
}The final FFmpeg pass then seeks both the video and voice assets to the same start point and resets timestamps. That keeps mouth movement, caption highlights, and audio impact aligned on frame zero.
ffmpeg \
-ss "$TRIM_SECONDS" -i avatar.mp4 \
-ss "$TRIM_SECONDS" -i voice.wav \
-filter_complex "[0:v]setpts=PTS-STARTPTS[v];[1:a]asetpts=PTS-STARTPTS[a]" \
-map "[v]" -map "[a]" \
-c:v libx264 -c:a aac hook-trimmed.mp4Why the audio tag bug mattered
PR #58 looked unrelated at first: an inline <audio> tag hid a late-starting hook and could be captured on screen. It was actually the same architectural problem. Browser preview and final render were sharing too much DOM. A visible or hidden audio element is convenient for preview controls, but a frame-capture renderer should only see visual layers.
The fix split preview state from render state. Preview can own playback controls. Render receives a visual scene graph plus an audio asset path. That same separation supports Dual Aspect-Ratio Rendering, where 9:16 and 16:9 frames are captured separately but share one canonical voice track.
Pipeline hardening PRs
| PR / branch | Problem | Fix |
|---|---|---|
#55 feat/trim-vo-lead-in |
Hook openers could spend the first 200-800ms in silence | Measured lead-in with FFmpeg, then trimmed and retimed output automatically |
#58 fix/audio-tag-blind-spot |
Inline audio tags made late-start issues invisible in preview and visible in capture | Separated preview playback controls from the render scene graph |
| #12 presenter follow-up | Presenter-generation assets had the same dead-air and catalog persistence risks | Ported dead-air trim into the presenter tool while preserving Drive-backed assets |
Operational guardrails
The trim pass should be automatic, but it still needs observability. The renderer records the measured lead-in, final trim, and whether the trim was skipped. That lets QA distinguish "no silence found" from "silence found but below threshold." For hook batches, the metadata also explains why one candidate feels sharper than another even when the prompt and voice are similar.
The key guardrail is idempotency. A user may regenerate a hook after a dead-air fix, or a worker may retry after FFmpeg fails. The trim result belongs in the job manifest so retries do not produce two slightly different timelines. I cover the paid-action side of that in Idempotency Transaction Guards & State Locks.
What I'd do differently
I would add client-side WebAudio analysis earlier as a preflight. Server-side FFmpeg is the source of truth for final render, but WebAudio could catch obvious dead air before upload or render dispatch, especially in voice-preview flows. That would save time without replacing the final server-side trim.
I would also make late-start metadata visible in the editor scorecard. Hook Machine already ranks candidates; a measured leadInMs signal belongs beside motion density and copy fit. Editors should see why a hook won, not just that it ranked first.
The pattern: self-healing rendering gates
Dead-air trimming is a small feature with a large product lesson: render pipelines should repair deterministic defects automatically. If a machine can measure the fault and the correction is safe, do not turn it into a manual QA burden. A 3-second hook has no room for preventable silence.