Applied AI and evaluationIn production (local)
Transcribing near-silent audio without inventing words
A local transcription stack that holds the leaderboard's best model back until an independent model confirms there is speech.
- Role
- Designer and evaluator
- Period
- Aug to Oct 2026
- Team
- Solo, with Claude Code
- Stack
- PythonSilero VADNVIDIA ParakeetCohere Transcribeffmpeg
Receipt Verified 2026-10-09whisper-large-v3 produced text on 93.5% of 200 clips where the voice detector found no speech.Period2026-08-01Method200 VAD-negative clips from field recordings; insertion rate per model.SourceBenchmark logCaveat"No speech" means the VAD found none, so this is an upper bound.of no-speech clips got Whisper text
Receipt Verified 2026-10-09Chose the enhancement chain with a pre-registered rule over 19,500 decodes; the negative control ranked last, as designed.Period2026-08-06Method24 recordings, 1,300 clips per rung, 5 signals x 3 models.SourceBenchmark logdecodes in a pre-registered bake-off
Receipt Verified 2026-10-09Per-recording selection moved 29 of 561 recordings to a minimal chain and recovered 28% more speech characters on them (2,023 to 2,590).PeriodSep 2026MethodCTC character counts on each recording's top windows; switch only on a >5% win.SourceSelection outputs (JSON)speech recovered where re-routed
Goal
Get reliable transcripts from long field recordings that are about 96% silence and HVAC noise, without any model filling the silence with text.
Obstacles
- Popular speech models invent text on non-speech: whisper-large-v3 produced text on Receipt Verified 2026-10-09whisper-large-v3 produced text on 93.5% of 200 clips where the voice detector found no speech.Period2026-08-01Method200 VAD-negative clips from field recordings; insertion rate per model.SourceBenchmark logCaveat"No speech" means the VAD found none, so this is an upper bound. of 200 clips with no detected speech.
- The most accurate transcriber was also the biggest fabricator by volume.
- Standard quality metrics reward over-processing: the setting with the cleanest-looking spectrum kept Receipt Verified 2026-10-09The rung with the best spectral score kept 5.9% fewer confirmed speech segments (3,371 vs 3,584) across 239 recordings.PeriodAug 2026MethodWhole-timeline re-transcription, paired by recording.SourceComparison report confirmed speech segments.
- No human-labeled ground truth exists for the real recordings.
Approach
- Voice activity detection first, so only candidate speech reaches a transcriber.
- A CTC model with no language prior holds a veto. If it hears nothing, the segment is rejected and logged as not transcribed.
- The leaderboard-best model transcribes only gated segments. A third decoder family scores agreement, and disagreements stay visible.
- A deterministic enhancement ladder, chosen by downstream recovered speech with a pre-registered rule over Receipt Verified 2026-10-09Chose the enhancement chain with a pre-registered rule over 19,500 decodes; the negative control ranked last, as designed.Period2026-08-06Method24 recordings, 1,300 clips per rung, 5 signals x 3 models.SourceBenchmark log decodes. The negative control ranked last, as designed.
- Per-recording chain selection: a recording switches only when a minimal chain wins by more than 5%. It moved 29 of 561 recordings and recovered Receipt Verified 2026-10-09Per-recording selection moved 29 of 561 recordings to a minimal chain and recovered 28% more speech characters on them (2,023 to 2,590).PeriodSep 2026MethodCTC character counts on each recording's top windows; switch only on a >5% win.SourceSelection outputs (JSON) speech on them.
- Chain of custody: SHA-256 of every source, Receipt Verified 2026-10-09751 WAV-to-FLAC conversions accepted only when the decoded audio's SHA-256 matched the original.PeriodOct 2026MethodSHA-256 of decoded PCM compared per file.SourceDriver log lossless conversions verified bit-exact, and a manifest over every output.
Lift (results)
- Receipt Verified 2026-10-09Ran the stack over 117.7 hours of audio across 267 recordings; 4.43 hours of it was speech.PeriodAug to Sep 2026MethodSum over per-recording summaries.SourceRun summaries (JSON) of audio across 267 recordings ran through the gated stack.
- A median Receipt Verified 2026-10-09The verified end-to-end chain (transfer, enhance, hash, verify, upload) took a median 22.4 minutes per 30-minute recording.PeriodOct 2026MethodDriver log timestamps; p10 9.9 min, p90 37.3 min.SourceDriver log per 30-minute recording, end to end: enhanced, hashed, verified and uploaded.
- On HVAC noise, enhancement cut the transcriber's fabrication rate from 24.06% → 13.44%.
What's actually new
No single algorithm here is new. The contribution is the evaluation system: an independent model with no language prior holds a veto, enhancement is scored by what the transcriber can't fake, and every control has to fail on demand before it is trusted.
Next
- A human-labeled gold set for a real word error rate and voice-detector recall.
- A clean ablation of neural denoisers on synthetic faint speech.
- Finish the CTC bench for the October batch.