-
feat: print transcription text in the binaries (embed vocab, in-binary detok) (#3004)
frostbyte_neo 发布于
2026-06-20 08:39:25 +00:00 SenseVoice and Paraformer binaries now output text directly (whisper-cli style),
no Python detok step.- export_{sensevoice,paraformer}_gguf.py: embed the vocab as a GGUF string array
(sv.vocab = SentencePiece pieces; pf.vocab = tokens.json) so the binary is self-contained. - funasr-sensevoice.cpp: CTC ids -> text (pieces + ▁->space), drop <|...|> meta by
default; --ids (raw), --keep-tags (keep meta tags). - funasr-paraformer.cpp: ids -> text (tokens.json, drop @@ continuations); --ids for raw.
- both fall back to ids if the GGUF has no vocab. READMEs updated to text output.
Verified: C++ text == reference detok (sentencepiece / tokens.json) on sample and the
full 002 clip; ids unchanged so CER is unchanged.下载附件
- export_{sensevoice,paraformer}_gguf.py: embed the vocab as a GGUF string array