and its square (default: (0.9, 0.999))
  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-8)

  * **weight_decay** (*float**, **optional*) -- weight decay (L2
    penalty) (default: 0)

  * **amsgrad** (*bool**, **optional*) -- whether to use the
    AMSGrad variant of this algorithm from the paper On the
    Convergence of Adam and Beyond (default: False)

  * **foreach** (*bool**, **optional*) -- whether foreach
    implementation of optimizer is used (default: None)

  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)

  * **capturable** (*bool**, **optional*) -- whether this instance
    is safe to capture in a CUDA graph. Passing True can impair
    ungraphed performance, so if you don't intend to graph capture
    this instance, leave it False (default: False)
