Yucheng Lu
b80e5624e2
Co-authored-by: Conglong Li <conglong.li@gmail.com> Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
29 lines
792 B
ReStructuredText
Executable File
29 lines
792 B
ReStructuredText
Executable File
Optimizers
|
|
===================
|
|
|
|
DeepSpeed offers high-performance implementations of ``Adam`` optimizer on CPU; ``FusedAdam``, ``FusedLamb``, ``OnebitAdam``, ``OnebitLamb`` optimizers on GPU.
|
|
|
|
Adam (CPU)
|
|
----------------------------
|
|
.. autoclass:: deepspeed.ops.adam.DeepSpeedCPUAdam
|
|
|
|
FusedAdam (GPU)
|
|
----------------------------
|
|
.. autoclass:: deepspeed.ops.adam.FusedAdam
|
|
|
|
FusedLamb (GPU)
|
|
----------------------------
|
|
.. autoclass:: deepspeed.ops.lamb.FusedLamb
|
|
|
|
OneBitAdam (GPU)
|
|
----------------------------
|
|
.. autoclass:: deepspeed.runtime.fp16.onebit.adam.OnebitAdam
|
|
|
|
ZeroOneAdam (GPU)
|
|
----------------------------
|
|
.. autoclass:: deepspeed.runtime.fp16.onebit.zoadam.ZeroOneAdam
|
|
|
|
OnebitLamb (GPU)
|
|
----------------------------
|
|
.. autoclass:: deepspeed.runtime.fp16.onebit.lamb.OnebitLamb
|