-
DeepCompile for enhanced compiler integration (#7154)
frostbyte_neo 发布于
2025-04-16 04:33:53 +00:00 This PR introduces DeepCompile, a new feature that efficiently
integrates compiler optimizations with other DeepSpeed features.
DeepCompile utilizes torch's dynamo to capture the computation graph and
modifies it to incorporate DeepSpeed’s optimizations seamlessly.Currently, DeepCompile supports ZeRO-1 and ZeRO-3, with enhancements
such as proactive prefetching and selective unsharding to improve
performance.
(More details will be added later.)
Signed-off-by: Masahiro Tanaka mtanaka@microsoft.com
Signed-off-by: Olatunji Ruwase olruwase@microsoft.com
Co-authored-by: zafarsadiq zafarsadiq120@gmail.com
Co-authored-by: Logan Adams 114770087+loadams@users.noreply.github.com
Co-authored-by: Olatunji Ruwase olruwase@microsoft.com下载附件