* feat(grpo): add RLSD self-distilled advantage reweighting
Introduce RLSD (Self-Distilled RLVR) token-level advantage reweighting for
GRPO, enabled via `--advantage_reweight rlsd`. The per-sequence advantage is
redistributed inside each trajectory using the teacher-vs-student logprob gap
as a sign-aware, clipped multiplicative weight, without flipping the reward
sign. The teacher reuses the OPSD self-distillation forward (current policy
conditioned on a privileged `teacher_prompt` column), and a frozen local
teacher via `--teacher_model` is also supported.
- add `apply_rlsd_reweight` in rl_core/advantage.py
- add RLSD args and lambda warmup/decay schedule to the GRPO trainer
- add `_check_rlsd` fail-fast validation in RLHFArguments
- add unit tests in tests/train/test_rlsd_reweight.py
* ci(workflows): add Qoder code review and assistant
* docs(grpo): document RLSD advantage reweighting parameters
Add the new RLSD (Self-Distilled RLVR) CLI parameters to the English and
Chinese command-line references: advantage_reweight, rlsd_lambda,
rlsd_reweight_clip_range, rlsd_lambda_warmup_steps, rlsd_lambda_decay_steps
and rlsd_negative_only, including defaults, valid ranges, the reweight
formula, enable requirements and the two teacher modes.
* update core-review.yaml
Added separate jobs for trusted and external code reviews in GitHub Actions workflow.
---------
Co-authored-by: jinghanhu <hujinghan.hjh@alibaba-inc.com>
QiyuanChen
ffb41eb082