Ahmadzei's picture
added 3 more tables for large emb model
5fa1a76
raw
history blame contribute delete
126 Bytes
DeepSpeed offers several optimizers (Adam, AdamW, OneBitAdam, and LAMB) but you can also import other optimizers from PyTorch.