Refresh the pinned llama.cpp revision with AMD Vulkan submission batching and DeviceLost diagnostics, adapt the sampler API, and document Windows troubleshooting. Refs #3479.
* feat(runtime): add Windows Vulkan release asset
* fix(ci): install SPIR-V headers for Windows Vulkan
* fix(runtime): make Windows Vulkan package standalone
Document the current sm_86 Windows CUDA prebuilt limit and direct RTX 50 / Blackwell users to CPU binaries or source builds with CMAKE_CUDA_ARCHITECTURES=120 until a dedicated asset is available.
Adds a Windows x64 CUDA release asset for the llama.cpp SenseVoice runtime, with CUDA Toolkit setup, architecture-86 build flags, focused SenseVoice target packaging, docs, and workflow regression tests. Manual workflow dispatch 29656771776 passed all build matrix jobs including build-windows-x64-cuda.
SenseVoice and Paraformer binaries now output text directly (whisper-cli style),
no Python detok step.
- export_{sensevoice,paraformer}_gguf.py: embed the vocab as a GGUF string array
(sv.vocab = SentencePiece pieces; pf.vocab = tokens.json) so the binary is self-contained.
- funasr-sensevoice.cpp: CTC ids -> text (pieces + ▁->space), drop <|...|> meta by
default; --ids (raw), --keep-tags (keep meta tags).
- funasr-paraformer.cpp: ids -> text (tokens.json, drop @@ continuations); --ids for raw.
- both fall back to ids if the GGUF has no vocab. READMEs updated to text output.
Verified: C++ text == reference detok (sentencepiece / tokens.json) on sample and the
full 002 clip; ids unchanged so CER is unchanged.
* test: numerical regression harness (frozen golden vs ggml/VAD/CIF/CTC output)
Adds tests/ — runs each runtime tool on a fixed 6 s clip and diffs against frozen
golden output, catching regressions in the ggml graphs, the FSMN-VAD state machine,
the CIF predictor and CTC decode.
- tests/run_regression.sh: auto-detects which tools are built; VAD model is auto-fetched
(1.7 MB), ASR GGUFs tested when present or with RUN_FULL=1 (downloads from HF).
Non-zero exit on any mismatch. BIN_DIR/MODELS_DIR overridable.
- tests/sample.wav (~6 s) + tests/golden/*.txt: golden captured on Linux x86-64 with the
f16 GGUFs from FunAudioLLM/*-GGUF.
Verified locally: all present tools PASS; default mode fetches VAD + skips absent models.
* test: golden as text (binaries now detok in-process; merge after in-binary-detok)
* test: fix sensevoice golden (correct transcription text)
- download-funasr-model.sh: detect `hf` (new CLI) or fall back to `huggingface-cli`
(older huggingface_hub); fail with an install hint if neither exists.
- convert-funasr-to-gguf.py: catch ModuleNotFoundError and print how to install the
missing package instead of a raw traceback.
Make the runtime a downloadable product (whisper.cpp-style), no manual export needed:
- download-funasr-model.sh: one command to pull pre-converted GGUF from Hugging Face
(FunAudioLLM/*-GGUF), incl. fsmn-vad for built-in --vad. No Python ML env required.
- convert-funasr-to-gguf.py: one-step HF/ModelScope checkpoint -> GGUF (wraps the
validated export_*.py; discovered relative to itself so it works in any layout).
- CMakeLists.txt: standalone, CI-friendly top-level build. FetchContent pins llama.cpp
(provides ggml + llama), static libs -> self-contained binaries in build/bin/, no
hardcoded paths; builds on Linux/macOS/Windows x64/arm64. `cmake -B build && cmake --build build -j`.
- README: download-first + standalone-build quickstart; links to the HF GGUF repos.
Verified on Linux: all targets build via FetchContent (static, no shared-lib deps);
hf download -> run reproduces canonical output. Additive; runtime/llama.cpp/ only.
- funasr_vad.h: define M_PI fallback (not guaranteed on MSVC) for cross-platform CI;
guard against audio shorter than one frame (empty fbank -> OOB in lfr);
validate required GGUF tensors up front (fail with a message, not a segfault);
shrink the no_alloc metadata context (512MB -> 16MB); drop a redundant ggml_cont
on an already-contiguous row slice in the FSMN memory loop.
- export_vad_gguf.py: use a with-statement when reading am.mvn (no fd leak).
No behavior change: VAD segment boundaries are byte-identical on the 184-clip set.
Add a native ggml FSMN-VAD front end so the binaries segment long audio themselves,
with no Python at runtime. New `--vad fsmn-vad.gguf` flag on llama-funasr-cli,
llama-funasr-sensevoice and llama-funasr-paraformer runs FSMN-VAD (fbank + LFR + CMVN +
FSMN encoder + E2EVadModel state machine, in funasr-common/funasr_vad.h) and decodes
each speech segment, then concatenates.
- funasr-common/funasr_vad.h: single-header VAD; segment boundaries within ~10 ms of
PyTorch fsmn-vad.generate on the 184-clip set (DEFAULT_SILENCE_SCHEDULE replicated,
chunk-stepped silence schedule + speech_noise_thres 0.5).
- funasr-vad/: standalone VAD tool (wav -> [start_ms,end_ms] segments).
- export_vad_gguf.py: export FSMN-VAD encoder + CMVN to GGUF (1.7 MB, 24 tensors).
- BENCHMARKS.md: bare binary with built-in VAD reaches the reference end-to-end —
full-184 micro-CER SenseVoice 8.01 / Paraformer 9.85 / Fun-ASR-Nano 8.30.
Additive only: no existing code is modified outside runtime/llama.cpp/.
* feat(runtime): accept any audio (any sample-rate/channels, wav/mp3/flac) via miniaudio
Replaces the 16k-mono-PCM16-only WAV reader with a miniaudio-based loader
(funasr-common/funasr_audio.h) that decodes wav/mp3/flac at any sample rate and
channel count, resampling to 16 kHz mono. Verified: a 44.1k stereo file and an 8k
mono file both transcribe correctly across cli/sensevoice/paraformer.
* fix(audio): inline loader (avoid duplicate symbols), null-path guard, check decoder read result (gemini review)
---------
Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
zhifu gao