# Speech sweep 2026-11-02

8 stitched long-form files from monsum20261029 (2-40 speakers, 8.3 h), `/v1/audio/transcriptions`
with `diarize=true`, no `language` (service auto-detects, as production does with mixed_language off).
Scored by `scripts/eval_speech.py score-diar`. "mn-only" = the 5 files the service decoded as Mongolian.

| Config | cpWER | WER | mean abs speaker err | mn-only cpWER | mn-only WER | wall | peak GPU |
| --- | --- | --- | --- | --- | --- | --- | --- |
| baseline | 0.524 | 0.468 | 3.38 | 0.294 | 0.212 | 24 m | 6.2 GB |
| DIARIZATION_EXCLUSIVE=1 | 0.523 | 0.468 | 3.38 | 0.293 | 0.212 | 25 m | 6.2 GB |
| WHISPER_DROP_HALLUCINATIONS=1 | 0.530 | 0.475 | 3.75 | 0.302 | 0.221 | 24 m | 6.2 GB |
| DIARIZATION_CLUSTERING_THRESHOLD=0.5 | 0.538 | 0.468 | 3.88 | 0.315 | 0.212 | 24 m | 6.2 GB |
| DIARIZATION_CLUSTERING_THRESHOLD=0.7 | 0.522 | 0.468 | 3.62 | 0.291 | 0.212 | 24 m | 6.2 GB |
| AmberNet language detection (lidfix) | 0.298 | 0.214 | 3.38 | n/a (8/8 mn) | n/a | 25 m | 4.3 GB |

Speakers ref>hyp per file (baseline): 2>3 4>4 40>21 8>7 24>20 10>10 12>12 6>8.

- 3/8 files (40-, 24-, 12-speaker) were auto-detected as English (p=0.514 > WHISPER_DETECT_MIN_PROB 0.5)
  and decoded by the stock model: cpWER ~1.0 on each. This dominates every overall number.
- Exclusive diarization and threshold 0.7 are within noise of baseline; 0.5 over-splits.
- Hallucination filter dropped 3-51 segments per file but raised WER: stitched gaps are digital silence,
  so there is little to remove and the filter cuts real speech. Needs real (unstitched) audio to judge.
- lidfix: request-level language detection moved from Whisper's token to AmberNet over speech spans;
  all 8 files now decode on the mn fine-tune. Per-file cpWER 0.16-0.45; worst are the 40-speaker (0.446)
  and 10-speaker (0.440) files.