Adadelta
class torch.optim.Adadelta(params, lr=1.0, rho=0.9, eps=1e-06, weight_decay=0, foreach=None, *, maximize=False, differentiable=False)
Implements Adadelta algorithm.
  \begin{aligned}      &\rule{110mm}{0.4pt}
  \\      &\textbf{input}      : \gamma \text{ (lr)}, \: \theta_0
  \text{ (params)},          \: f(\theta) \text{ (objective)}, \:
  \rho \text{ (decay)},          \: \lambda \text{ (weight decay)}
  \\      &\textbf{initialize} :  v_0  \leftarrow 0 \: \text{
  (square avg)},          \: u_0 \leftarrow 0 \: \text{
  (accumulate variables)}                     \\[-1.ex]
  &\rule{110mm}{0.4pt}
  \\      &\textbf{for} \: t=1 \: \textbf{to} \: \ldots \:
  \textbf{do}                         \\      &\hspace{5mm}g_t
  \leftarrow   \nabla_{\theta} f_t (\theta_{t-1})           \\
  &\hspace{5mm}if \: \lambda \neq 0
  \\      &\hspace{10mm} g_t \leftarrow g_t + \lambda
  \theta_{t-1}                            \\      &\hspace{5mm}
