and its square (default: (0.9, 0.999))
  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-8)

  * **weight_decay** (*float**, **optional*) -- weight decay
    coefficient (default: 1e-2)

  * **amsgrad** (*bool**, **optional*) -- whether to use the
    AMSGrad variant of this algorithm from the paper On the
    Convergence of Adam and Beyond (default: False)

  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)

  * **foreach** (*bool**, **optional*) -- whether foreach
    implementation of optimizer is used. If unspecified by the
    user (so foreach is None), we will try to use foreach over the
    for-loop implementation on CUDA, since it is usually
    significantly more performant. (default: None)

  * **capturable** (*bool**, **optional*) -- whether this instance
