class_docstr: Weight Standardization (WS) link hook implementation.
  
  This hook standardizes a weight by *weight statistics*.
  
  This link hook implements a WS which computes the mean and variance along
  axis "output channels", then normalizes by these statistics.
  WS improves training by reducing the Lipschitz constants of the loss and
  the gradients like batch normalization (BN) but without relying on large
  batch sizes during training. Specifically, the performance of WS with group
  normalization (GN) trained with small-batch is able to match or outperforms
  that of BN trained with large-batch.
  WS is originally proposed for 2D convolution layers followed by mainly GN
  and sometimes BN.
  Note that this hook is able to handle layers such as N-dimensional
  convolutional, linear and embedding layers but there is no guarantee that
  this hook helps training.
  
  See: Siyuan Qiao et. al., `Weight Standardization
  <https://arxiv.org/abs/1903.10520>`_
  
  Args:
      eps (float): Numerical stability in standard deviation calculation.
          The default value is 1e-5.
      weight_name (str): Link's weight name to appky this hook. The default
          value is ``'W'``.
      name (str or None): Name of this hook. The default value is
          ``'WeightStandardization'``.
class_docstr_tok: ["Weight","Standardization","(","WS",")","link","hook","implementation",".","This","hook","standardizes","a","weight","by","*","weight","statistics","*",".","This","link","hook","implements","a","WS","which","computes","the","mean","and","variance","along","axis","``","output","channels","''",",","then","normalizes","by","these","statistics",".","WS","improves","training","by","reducing","the","Lipschitz","constants","of","the","loss","and","the","gradients","like","batch","normalization","(","BN",")","but","without","relying","on","large","batch","sizes","during","training",".","Specifically",",","the","performance","of","WS","with","group","normalization","(","GN",")","trained","with","small-batch","is","able","to","match","or","outperforms","that","of","BN","trained","with","large-batch",".","WS","is","originally","proposed","for","2D","convolution","layers","followed","by","mainly","GN","and","sometimes","BN",".","Note","that","this","hook","is","able","to","handle","layers","such","as","N-dimensional","convolutional",",","linear","and","embedding","layers","but","there","is","no","guarantee","that","this","hook","helps","training",".","See",":","Siyuan","Qiao","et",".","al.",",","`","Weight","Standardization","<","https",":","\/\/arxiv.org\/abs\/1903.10520",">","`","_","Args",":","eps","(","float",")",":","Numerical","stability","in","standard","deviation","calculation",".","The","default","value","is","1e-5",".","weight_name","(","str",")",":","Link","'s","weight","name","to","appky","this","hook",".","The","default","value","is","``","'","W","'","``",".","name","(","str","or","None",")",":","Name","of","this","hook",".","The","default","value","is","``","'WeightStandardization","'","``","."]
code: def __init__(self, *, eps=1e-5, weight_name="W", name=None):
      self.eps = eps
      self.weight_name = weight_name
      self._initialized = False
      if name is not None:
          self.name = name
  
file_path: chainer/chainer/link_hooks/weight_standardization.py
imports: import chainer
  from chainer.functions.normalization import group_normalization
  from chainer import link_hook
label_desc: Use image node_id 1 to create a new WeightStandardization object from inherited base classes: link_hook with example: obj = WeightStandardization()