Streaming transcription, speaker labels and endpointing from open weights on a CPU
Get what Meta's Muse Voice Transcribe does from one closed model, words, speaker labels, endpoints, out of separate open components, on a CPU, with the audio never leaving the machine.
The models were the easy part and they are swappable. The problem is that the
three signals arrive on three different latencies, and a closed model that emits
them as tokens in one vocabulary gets the ordering for free. So the project is
the merge. The rule that makes it usable is that committed output is append
only: once a commit is emitted nothing later contradicts it. A speaker label
cannot exist before enough audio has been heard, so early words commit with
speaker: null and a later event fills them in retroactively. Committing a
guess there would be unfixable; leaving a gap is not. A consumer that ignores
every partial still gets a correct transcript, which is what keeps simple
integrations simple.
And installable as hearwrite[onnx,turn,server]. Measured on an M4, CPU only:
emission delay p50 0.28s to 0.52s, false endpoint on 5.0% of mid-thought pauses,
6.8% and 4.4% WER on LibriSpeech dev-clean, all reproducible from
docs/evaluation.md. Diarization is on the front page as experimental, because
three people on one laptop microphone finds two speakers and splits turns mid
sentence. Still active.