class_docstr: Update rule of the M-SVAG optimization algorithm.
  
  See: `Dissecting Adam: The Sign, Magnitude and Variance of Stochastic
  Gradients <https://arxiv.org/abs/1705.07774>`_
  
  Modified for proper weight decay.
  
  See: `Fixing Weight Decay Regularization in Adam
  <https://openreview.net/forum?id=rk6qdGgCZ>`_
  
  See :class:`~chainer.optimizers.MSVAG` for the default values
  of the hyperparameters.
  
  Args:
      parent_hyperparam (~chainer.optimizer.Hyperparameter): Hyperparameter
          that provides the default values.
      lr (float): Learning rate.
      beta (float): Exponential decay rate of the first and second order
                    moment.
      eta (float): Schedule multiplier, can be used for warm restarts.
      weight_decay_rate (float): Weight decay rate.
code: def update_core_gpu(self, param):
      grad = param.grad
      if grad is None:
          return
  
      hp = self.hyperparam
  
      rho = (
          ((1.0 - hp.beta) ** 2)
          * (1.0 - self.beta_power**2)
          / (((1.0 - self.beta_power) ** 2) * (1.0 - hp.beta**2))
      )
      rho = min(rho, 0.9999)
  
      cuda.elementwise(
          "T grad, T lr, T one_minus_beta, T eta, \
               T weight_decay_rate, T beta_power, T rho",
          "T param, T m, T v",
          """m += one_minus_beta * (grad - m);
                 v += one_minus_beta * (grad * grad - v);
  
                 T mt = m / (1.0 - beta_power);
                 T vt = v / (1.0 - beta_power);
  
                 T mt2 = mt*mt;
                 T s = (vt - mt2) / (1.0 - rho);
  
                 T factor;
                 if (m == 0 && v == 0)
                     factor = 0.0;
                 else
                     factor = min(1.0, max(0.0, mt2 / (mt2 + rho * s)));
  
                 param -= eta * (lr * mt * factor +
                                 weight_decay_rate * param);""",
          "msvag",
      )(
          grad,
          hp.lr,
          1 - hp.beta,
          hp.eta,
          hp.weight_decay_rate,
          self.beta_power,
          rho,
          param.data,
          self.state["m"],
          self.state["v"],
      )
  
      self.beta_power *= hp.beta
  
file_path: chainer/chainer/optimizers/msvag.py
imports: from __future__ import division
  import numpy
  import chainer
  from chainer.backends import cuda
  from chainer import optimizer
  from chainer import types
label_desc: Use image node_id 4 for calling the MSVAGRule obj's underlying member method code with example usage: obj.update_core_gpu(param) without return types