inkcherry
66af8f03a3
## Summary RFC: https://github.com/deepspeedai/DeepSpeed/issues/7884 Wire `sdma_allgather` into ZeRO-3's parameter prefetch path (`_dist_allgather_fn`). When enabled, ZeRO-3 allgather routes through `mori_cpp.AllGatherIntoTensor` (intra-node SDMA copy on AMD MI300), with a transparent fallback to `dist.allgather_fn` (RCCL/NCCL) on init failure. End-to-end demo + repro steps + verified numbers live in [`examples/sdma_allgather/README.md`](examples/sdma_allgather/README.md). Headline (8x MI300X, DeepSpeed default ZeRO-3 buckets, 100 steps): | | GPT-7B-ish | Qwen3-32B | |---|---|---| | SDMA off | 697.7 ms / step | 1402.5 ms / step | | SDMA on | 622.0 ms / step | 1263.2 ms / step | | **gain** | **+10.85 %** | **+9.93 %** | Loss curves match off ↔ on, peak memory unchanged. Speedup is workload-dependent — gains shrink (or invert) when allgather can't be overlapped with compute Co-authored-by: wuyl1 <yangwu@amd.com> --------- Signed-off-by: wuyl1 <yangwu@amd.com> Signed-off-by: inkcherry <mingzhi.liu@amd.com> Co-authored-by: wuyl1 <yangwu@amd.com>
DeepSpeed Examples
If you are looking for examples using DeepSpeed please see the following resources: