on for more configuration options. You should also change the default Adam optimizer to DeepSpeed’s optimized version of Adam deepspeed.ops.adam.DeepSpeedCPUAdam for a substantial speedup. Enabling DeepSpeedCPUAdam requires your system’s CUDA toolchain version to be the same as the one installed with PyTorch. bitsandbytes 8-bit optimizers don’t seem to be compatible with DeepSpeed at the moment. That’s it! You don’t need to add any additional p