class_docstr: Update rule of the M-SVAG optimization algorithm.
  
  See: `Dissecting Adam: The Sign, Magnitude and Variance of Stochastic
  Gradients <https://arxiv.org/abs/1705.07774>`_
  
  Modified for proper weight decay.
  
  See: `Fixing Weight Decay Regularization in Adam
  <https://openreview.net/forum?id=rk6qdGgCZ>`_
  
  See :class:`~chainer.optimizers.MSVAG` for the default values
  of the hyperparameters.
  
  Args:
      parent_hyperparam (~chainer.optimizer.Hyperparameter): Hyperparameter
          that provides the default values.
      lr (float): Learning rate.
      beta (float): Exponential decay rate of the first and second order
                    moment.
      eta (float): Schedule multiplier, can be used for warm restarts.
      weight_decay_rate (float): Weight decay rate.
code: def update_core_cpu(self, param):
      grad = param.grad
      if grad is None:
          return
      hp = self.hyperparam
      m, v = self.state["m"], self.state["v"]
  
      rho = (
          ((1.0 - hp.beta) ** 2)
          * (1.0 - self.beta_power**2)
          / (((1.0 - self.beta_power) ** 2) * (1.0 - hp.beta**2))
      )
      rho = min(rho, 0.9999)
  
      m += (1 - hp.beta) * (grad - m)
      v += (1 - hp.beta) * (grad * grad - v)
  
      mt = m / (1 - self.beta_power)
      vt = v / (1 - self.beta_power)
  
      mt2 = mt**2
      s = (vt - mt2) / (1 - rho)
  
      factor = numpy.clip(mt2 / (mt2 + rho * s), 0, 1)
      if isinstance(factor, numpy.ndarray):
          factor[numpy.isnan(factor)] = 0
      else:
          if numpy.isnan(factor):
              factor = 0
  
      param.data -= hp.eta * (
          hp.lr * mt * factor + hp.weight_decay_rate * param.data
      )
  
      self.beta_power *= hp.beta
  
file_path: chainer/chainer/optimizers/msvag.py
imports: from __future__ import division
  import numpy
  import chainer
  from chainer.backends import cuda
  from chainer import optimizer
  from chainer import types
label_desc: Use image node_id 3 for calling the MSVAGRule obj's underlying member method code with example usage: obj.update_core_cpu(param) without return types