Back to Blog

Voice Pipeline Economics: Double-Billing, a Backwards Ladder, and a Lexicon That Never Reached the Voice

By · 9 min read
Voice AI Kling Lip Sync Cost Engineering TypeScript

Every AI video pipeline eventually has to answer an unglamorous question: what did we actually pay for that clip? On the main video-generation service, the answer for months had been "a hardcoded constant." That's fine until the vendor changes its own pricing, or a code path pays for the same synthesis twice, or a voice engine mints a clone, bills for it, and never sends it downstream. Over a ten-PR run I audited and rebuilt the voice and lip-sync pipeline from the billing layer up, then used the vendor's own SKU tiers to cut cost 7x without touching output quality.

A cost model built from hardcoded constants isn't a cost model. It's a guess that happens to compile.

Billing what the vendor actually charges

PR #224 was workstream one of three from a sibling-tool audit: port the cost-accounting fixes that Presenter Generation and Variant Multiplier had already found, verifying each one against this repo's own code rather than assuming the same defect existed in the same place. Anthropic returns exact token counts on every response. Nothing in the pipeline read them — every charge was a hardcoded per-call constant, so the ledger and the vendor invoice diverged the moment usage drifted from whatever number had been typed in at launch.

The same PR closed a second gap: two editor-facing routes could spend money — kicking off a generation, retrying a step — outside any run. A run is the unit everything else (budgets, audit trail, the cost ledger) is keyed to. A spend with no run attached is a spend the ledger can't even see, which is worse than a wrong number.

Paying twice for a take the model returns unchanged

PR #225 found the sibling bug's twin: some vendor calls return the exact same asset on a retry — no new synthesis happened — and the pipeline billed a second time anyway because "call succeeded" and "call did new work" were treated as the same fact. The fix is the boring, correct kind: hash the output, and only charge when the hash changes from the take you already paid for.

A budget/quality SKU ladder that opened backwards

PR #232 replaced one hardcoded lip-sync SKU pair with two explicit fallback chains:

ChainLadderRate
budget (default)1.9.0-beta → v2$0.0117/s → $0.05/s
quality (rollback)v2/pro → react-1$0.0833/s → $0.167/s

That's a 7x cut against v2/pro on the pass, but the ladder direction is what makes it safe rather than reckless. Both chains fall back upward — to a better SKU than they opened with, never a worse one. A fallback fires because the first choice failed; degrading further would trade an outage for a silent quality drop nobody chose. react-1 only appears in the quality chain, because landing there by accident from the budget chain costs 14x the cheapest SKU with nobody having decided to pay for it.

A cache keyed on the wrong thing

PR #229 is the pattern this whole cluster keeps rediscovering in different clothes: a cache key must move when its content does. The voiceover cache was keyed on scene index — vo-scenes/<idx>.mp3 either existed or it didn't. Changing the voice, the scene's emotion, the TTS model, or any of three delivery flags left the old recording in place, generated under settings that no longer applied. Only a text edit invalidated it, because invalidateSceneVo was called from exactly two places in the whole codebase.

The fix content-keys the cache via a sidecar file rather than renaming the asset itself — the filename can't move because it's served over HTTP by path — and makes the warp chain that depends on it optional rather than assumed-present.

Minted, billed, and never sent

PR #258 is the sharpest failure in the batch. A Kling voice clone was minted from the source audio, billed to the run's ledger, and then never actually sent to the clip-generation call that was supposed to use it — the clip rendered with Kling's default voice while the ledger recorded a custom-clone charge. The bug produced a video that sounded fine, which is exactly why nobody caught it by watching output: the defect was invisible in the artifact and only visible in the diff between the request built and the request sent.

// before: clone minted, billed, and dropped on the floor
const clone = await mintKlingVoiceClone(audioBuffer);
await ledger.charge(runId, 'voice_clone', clone.cost);
// clip request never referenced clone.voiceId — Kling used its own default

// after: the clone id is a required field on the clip request,
// not an optional one the caller can forget to attach
const clip = await requestKlingClip({
  ...clipParams,
  voiceId: clone.voiceId, // TypeScript now refuses to compile without this
});

A voice bind that can 422 a whole run

PR #260 found that the voice-bind call — attaching a cloned voice to a specific Kling model invocation — could 422 and take down every clip in the run with it, not just the one clip that needed the voice. A probe surfaced the failure mode: bind failures need to degrade one clip, not cascade to the batch. PR #271 found the same class of bug from the other direction — every scene ended up with a different voice because the clone was minted from a source file that didn't exist yet at mint time, a race between file-write and clone-request that only showed up under real concurrency.

A pronunciation dictionary nobody could hear

The scripts for this pipeline are built on product-specific vocabulary — anatomical terms, brand names — and a TTS voice guesses at pronunciation differently take to take, so the same word could be said two different ways inside a single video. PR #230 built a pronunciation dictionary and, more importantly, a test that could actually fail against it, because "the audio came back different" proves nothing against a non-deterministic voice model on its own.

That dictionary shipped and did nothing for two more PRs. PR #262, on the sibling repo's own pipeline, found the lexicon was attached to a voice synthesis call nobody actually listened to downstream — the dictionary was correct, the wiring wasn't. PR #261 refactored the rule shape first, so a pronunciation rule could carry either a phoneme or a plain-text alias, because some mispronunciations aren't phonetic at all — they're the model reading a brand name as an acronym.

The pattern across all ten PRs

Every fix in this cluster is a variant of the same root cause: a value computed correctly in one place and consumed incorrectly — or not at all — somewhere downstream. The billing constant was correct until the vendor's pricing moved. The cache key was correct until content changed without the key changing. The voice clone was correctly minted and billed and then silently dropped before the request that needed it. None of these are algorithmically hard bugs. They're wiring bugs, and the only defense against wiring bugs is measuring the actual request sent, not the intent that produced it — which is why almost every PR here ends with a test or a probe that reads the real payload, not the code that built it.

Related Articles