SparseAdam
class torch.optim.SparseAdam(params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, maximize=False)
Implements lazy version of Adam algorithm suitable for sparse
   tensors.
In this variant, only moments that show up in the gradient get
   updated, and only those portions of the gradient get applied to the
   parameters.
Parameters:
      * params (iterable) -- iterable of parameters to optimize
        or dicts defining parameter groups
  * **lr** (*float**, **optional*) -- learning rate (default:
    1e-3)

  * **betas** (*Tuple**[**float**, **float**]**, **optional*) --
    coefficients used for computing running averages of gradient
    and its square (default: (0.9, 0.999))

  * **eps** (*float**, **optional*) -- term added to the
    denominator to improve numerical stability (default: 1e-8)

  * **maximize** (*bool**, **optional*) -- maximize the params
    based on the objective, instead of minimizing (default: False)
