Adagrad
class torch.optim.Adagrad(params, lr=0.01, lr_decay=0, weight_decay=0, initial_accumulator_value=0, eps=1e-10, foreach=None, *, maximize=False, differentiable=False)
Implements Adagrad algorithm.
  \begin{aligned}      &\rule{110mm}{0.4pt}
  \\      &\textbf{input}      : \gamma \text{ (lr)}, \: \theta_0
  \text{ (params)}, \: f(\theta)          \text{ (objective)}, \:
  \lambda \text{ (weight decay)},                          \\
  &\hspace{12mm}    \tau \text{ (initial accumulator value)}, \:
  \eta\text{ (lr decay)}\\      &\textbf{initialize} :
  state\_sum_0 \leftarrow 0                             \\[-1.ex]
  &\rule{110mm}{0.4pt}
  \\      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
  \textbf{do}                         \\      &\hspace{5mm}g_t
  \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \\
  &\hspace{5mm} \tilde{\gamma}    \leftarrow \gamma / (1 +(t-1)
