Release FunASR 1.4.3 with the optional Silero VAD adapter and scalable fixed-K diarization clustering.\n\nSigned-off-by: zhifu gao <18321252+LauraGPT@users.noreply.github.com>
Resolve distributed trainer settings once with top-level precedence and backwards-compatible train_conf fallbacks. Remove consumed mode keys before expanding trainer_conf, reject simultaneous FSDP and DeepSpeed, and cover the behavior with focused tests.
Preserve subtitle segmentation when punctuation token boundaries split a timestamped ASR word.
Signed-off-by: LauraGPT <LauraGPT@users.noreply.github.com>
The tests merged in #3456 were flaky: the state-dict round-trip test draws from
the unseeded global RNG, and the merged vs unmerged fp32 forwards can differ by
~4e-5 (different op orderings), above torch.allclose's default 1e-5 rtol. The
test failed on roughly one in three full-file runs.
- Seed the RNG per test so every test is reproducible.
- Compare merge/unmerge at a fp32-appropriate rtol=1e-3. A genuinely broken
merge differs by the full adapter term (O(1-10)), not ~1e-5, so the tolerance
is still discriminating (verified: a 'mark merged but don't fold' bug yields
max_abs_diff ~12).
- Add test_mark_only_lora_as_trainable_keeps_adapters_trainable, which exercises
the lora_only=true flow (mark_only_lora_as_trainable in train.py/train_ds.py)
that the previous reply described but did not actually test.
Verified: 8/8 pass, stable across repeated runs; the full nano set is 35 passed.
Co-authored-by: Claude <noreply@anthropic.com>
* feat(nano): support LoRA finetuning of the Qwen3 LLM
The FunASR-Nano model config already declares llm_conf.use_lora and
llm_conf.lora_conf (r/lora_alpha/lora_dropout/target_modules), but nothing
consumed them: use_lora was silently ignored and training fell back to the
full-parameter LLM path. Wire it up:
- When llm_conf.use_lora is true, replace the LLM target Linear layers
(default q_proj/v_proj) with lora.Linear adapters, sharing the frozen base
weight and adding trainable lora_A/lora_B. The adapter params are created in
the base weight's dtype (bf16), so the LoRA path does not depend on autocast.
- Pairs with the existing lora_only / mark_only_lora_as_trainable flow in
train.py / train_ds.py: lora_only=true freezes every non-LoRA parameter for
pure-LoRA training; lora_only=false + unfrozen encoder/adaptor conf keeps
them trainable while LoRA-tweaking only the LLM.
- Checkpoints save the full state dict (unchanged base + adapter params), so a
LoRA run can be resumed or decoded with the same use_lora=true config; for a
standalone deployment checkpoint, fold with W' = W + alpha/r * B @ A.
- Add lora_finetune.sh and document the recipe in finetune.md / finetune_zh.md.
Validated end-to-end with the real model: 56 adapters injected (28 Qwen3
layers x q_proj+v_proj), optimizer updates only the 112 LoRA params, and a
2-step smoke training run through funasr-train-ds completes and saves a
checkpoint whose lora keys are bf16 and match the base dtype.
Co-Authored-By: Claude <noreply@anthropic.com>
* test(nano): add offline unit tests for LoRA injection
Drive FunASRNano._apply_lora_to_llm against a tiny fake LLM mirroring the
Qwen3 parameter layout (layers.<i>.self_attn.{q,k,v,o}_proj / mlp.{gate,up,down}_proj).
No model download needed.
Covers the points requested in review:
1. only configured target modules are replaced;
2. the initial output matches the original linear layers (lora_B init zero);
3. base weights stay frozen while lora_A/lora_B are trainable (llm_conf.freeze
applied before injection, as in the real init path);
4. a non-zero adapter survives a state-dict save/load round trip and train/eval
transitions (including the eval-merge / train-unmerge convention);
5. the no-matching-target configuration is an explicit no-op with a warning.
Verified honest: on upstream main (no _apply_lora_to_llm) all 7 fail with
AttributeError; with the fix 7/7 pass. Full nano set: 34 passed.
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix(trainer): avoid KeyError when a checkpoint is saved at an unvalidated step
When save_checkpoint_interval is not a multiple of validate_interval, a
checkpoint can be saved at a step that was never validated. The best-ckpt
comparison then indexes val_acc/val_loss_step_or_epoch[ckpt_name], raising
KeyError right after the checkpoint was already written and aborting the run.
Guard both the DeepSpeed and the torch.save checkpoint paths with .get() so
an unvalidated checkpoint is skipped for the "best" bookkeeping (and logged)
instead of crashing; fall back to 0.0 when recording saved_ckpts.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(trainer): exclude unvalidated-step checkpoints from best-model ranking
Address review feedback on PR #3455. A checkpoint saved at a step that was
never validated (save_checkpoint_interval not a multiple of
validate_interval) used to be inserted into `saved_ckpts` with a fabricated
score of 0.0. Under loss-based ranking that synthetic "perfect" loss could
evict a real validated best checkpoint via keep_nbest_models pruning.
Now such checkpoints are kept on disk but excluded from `saved_ckpts`
entirely, so they never compete in best-model ranking or pruning, and
`best_step_or_epoch` only ever references validated checkpoints.
Adds regression coverage for both loss and acc ranking on the DeepSpeed
and torch.save paths, asserting an unvalidated checkpoint cannot evict a
validated best checkpoint.
* test(trainer): make unvalidated-ckpt regression catch the DeepSpeed+acc case
The fake DeepSpeed engine now writes a placeholder file per tag (mirroring
DeepSpeed's per-tag checkpoint dirs), so keep_nbest_models pruning's
smart_remove() is observable. This closes a gap where the [DeepSpeed, acc]
combination passed on the buggy code: under acc ranking the fabricated 0.0
is the minimum, so pruning deletes the unvalidated checkpoint itself rather
than the validated best, and without an on-disk artifact that deletion was
invisible.
With the fix, all 8 cases pass; on the previous code all 4 unvalidated-step
cases now fail (loss: fabricated 0.0 evicts the validated best; acc: the
unvalidated checkpoint file is pruned).
* fix(trainer): apply the same unvalidated-step guard to the non-DeepSpeed Trainer
`funasr-train` (funasr/train_utils/trainer.py) shares the identical
checkpoint-ranking code as trainer_ds.py and had the same bug: direct
indexing of `val_{acc,loss}_step_or_epoch[ckpt_name]` raised KeyError when
a checkpoint was saved at a step that was never validated, and the
keep_nbest_models bookkeeping inserted a fabricated score for such
checkpoints.
Apply the same fix: checkpoints saved at an unvalidated step are kept on
disk but excluded from saved_ckpts (no synthetic score), best_step_or_epoch
only ever references validated checkpoints, and the metric dicts are read
via .get().
Tests now parametrize over all three checkpointing paths (trainer_ds
torch.save, trainer_ds DeepSpeed, and trainer torch.save) x loss/acc.
On upstream main all 6 unvalidated-step cases fail with
`KeyError: 'model.pt.ep1.2'`; with the fix all 12 cases pass.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Include SenseVoice English normalization data and RWKV-BAT CUDA kernel sources in PyPI artifacts, with a real-wheel release contract.\n\nSigned-off-by: zhifu gao <18321252+LauraGPT@users.noreply.github.com>
Advance the package version to 1.4.0, document the Python and tagged-source release changes in four languages, and enforce the release numbering contract.
Signed-off-by: LauraGPT <lauragpt@users.noreply.github.com>
Fail immediately with an actionable parameter name when `vda_model` is supplied, instead of silently disabling the VAD-dependent long-audio and speaker pipeline.
Signed-off-by: Zhifu Gao <gaozhifu.gzf@alibaba-inc.com>
Bind the public homepage patrol contract to the canonical product-site build so copy changes cannot leave the monitor reporting a false production regression.
Signed-off-by: Zhifu Gao <gaozhifu.gzf@alibaba-inc.com>
Decode encoded audio containers before inference while preserving the raw PCM fast path. Rewind file-like decoder fallbacks and support ffmpeg stdin decoding.
Signed-off-by: LauraGPT <lauragpt@users.noreply.github.com>
Fall back to VAD-aligned string sentence records when a sized punctuation array cannot align with long-audio ASR timestamps.
Verified against both reporter attachments: 103.mp3 recovers 199 sentence records and 2980.mp3 remains byte-for-byte unchanged.
Signed-off-by: LauraGPT <lauragpt@users.noreply.github.com>
Make spk=true functional across vLLM and fallback server paths, preserve speaker labels in verbose_json segments, and accept sentence_info sentence text.
Discover all public HTML pages through the sitemap and enforce the bilingual donor entry as the final directory link. Includes regression coverage for missing, misplaced, duplicate, and wrong-language links.
zhifu gao