class_docstr: Batch normalization module.
  
  This implements normalization across the batch and spatial dimensions.
  It maintains moving averages of the mean and variance which can be
  used to normalize at test time. The constructor is generic and
  requires the user to pass in objects to compute these.
  
  At training time we use the batch statistics for that batch and these are then
  used to update the moving averages.
  
  At test time we can either use the moving averages of the batch statistics
  (``test_local_stats=False``) or we can use the local statistics
  (``test_local_stats=True``).
  
  It transforms the input ``x`` into:
  
  .. math::
  
      \d{outputs} = \d{scale} \dfrac{x - \mu}{\sigma + \epsilon} + \d{offset}
  
  Where :math:`\mu` and :math:`\sigma` are respectively the mean and standard
  deviation of ``x``. Note that this module automatically uses the fused batch
  norm op if the data format is ``NHWC``.
  
  There are many different variations for how users want to manage scale and
  offset if they require them at all. These are:
  
    - No scale/offset in which case ``create_*`` should be set to ``False`` and
      ``scale``/``offset`` aren't passed when the module is called.
    - Trainable scale/offset in which case ``create_*`` should be set to
      ``True`` and again ``scale``/``offset`` aren't passed when the module is
      called. In this case this module creates and owns the ``scale``/``offset``
      variables.
    - Externally generated ``scale``/``offset``, such as for conditional
      normalization, in which case ``create_*`` should be set to ``False`` and
      then the values fed in at call time.
  
  Attributes:
    scale: If ``create_scale``, a trainable :tf:`Variable` holding the current
      scale after the module is connected for the first time.
    offset: If ``create_offset``, a trainable :tf:`Variable` holding the current
      offset after the module is connected for the first time.
class_docstr_tok: ["Batch","normalization","module",".","This","implements","normalization","across","the","batch","and","spatial","dimensions",".","It","maintains","moving","averages","of","the","mean","and","variance","which","can","be","used","to","normalize","at","test","time",".","The","constructor","is","generic","and","requires","the","user","to","pass","in","objects","to","compute","these",".","At","training","time","we","use","the","batch","statistics","for","that","batch","and","these","are","then","used","to","update","the","moving","averages",".","At","test","time","we","can","either","use","the","moving","averages","of","the","batch","statistics","(","``","test_local_stats=False","``",")","or","we","can","use","the","local","statistics","(","``","test_local_stats=True","``",")",".","It","transforms","the","input","``","x","``","into",":","..","math",":",":","\\d","{","outputs","}","=","\\d","{","scale","}","\\dfrac","{","x","-","\\mu","}","{","\\sigma","+","\\epsilon","}","+","\\d","{","offset","}","Where",":","math",":","`","\\mu","`","and",":","math",":","`","\\sigma","`","are","respectively","the","mean","and","standard","deviation","of","``","x","``",".","Note","that","this","module","automatically","uses","the","fused","batch","norm","op","if","the","data","format","is","``","NHWC","``",".","There","are","many","different","variations","for","how","users","want","to","manage","scale","and","offset","if","they","require","them","at","all",".","These","are",":","-","No","scale\/offset","in","which","case","``","create_","*","``","should","be","set","to","``","False","``","and","``","scale","``","\/","``","offset","``","are","n't","passed","when","the","module","is","called",".","-","Trainable","scale\/offset","in","which","case","``","create_","*","``","should","be","set","to","``","True","``","and","again","``","scale","``","\/","``","offset","``","are","n't","passed","when","the","module","is","called",".","In","this","case","this","module","creates","and","owns","the","``","scale","``","\/","``","offset","``","variables",".","-","Externally","generated","``","scale","``","\/","``","offset","``",",","such","as","for","conditional","normalization",",","in","which","case","``","create_","*","``","should","be","set","to","``","False","``","and","then","the","values","fed","in","at","call","time",".","Attributes",":","scale",":","If","``","create_scale","``",",","a","trainable",":","tf",":","`","Variable","`","holding","the","current","scale","after","the","module","is","connected","for","the","first","time",".","offset",":","If","``","create_offset","``",",","a","trainable",":","tf",":","`","Variable","`","holding","the","current","offset","after","the","module","is","connected","for","the","first","time","."]
docstr: Constructs a ``BaseBatchNorm`` module.
  
  Args:
    create_scale: whether to create a trainable scale per channel applied
      after the normalization.
    create_offset: whether to create a trainable offset per channel applied
      after normalization and scaling.
    moving_mean: A metric which tracks the moving average of the mean which
      can be used to normalize at test time.
    moving_variance: A metric which tracks the moving average of the variance
      which can be used to normalize at test time.
    eps: Small epsilon to avoid division by zero variance. Defaults to
      ``1e-5``.
    scale_init: Optional initializer for the scale variable. Can only be set
      if ``create_scale=True``. By default scale is initialized to ``1``.
    offset_init: Optional initializer for the offset variable. Can only be set
      if ``create_offset=True``. By default offset is initialized to ``0``.
    data_format: The data format of the input. Can be either
      ``channels_first``, ``channels_last``, ``N...C`` or ``NC...``. By
      default it is ``channels_last``.
    name: Name of the module.
docstr_tok: ["Constructs","a","``","BaseBatchNorm","``","module",".","Args",":","create_scale",":","whether","to","create","a","trainable","scale","per","channel","applied","after","the","normalization",".","create_offset",":","whether","to","create","a","trainable","offset","per","channel","applied","after","normalization","and","scaling",".","moving_mean",":","A","metric","which","tracks","the","moving","average","of","the","mean","which","can","be","used","to","normalize","at","test","time",".","moving_variance",":","A","metric","which","tracks","the","moving","average","of","the","variance","which","can","be","used","to","normalize","at","test","time",".","eps",":","Small","epsilon","to","avoid","division","by","zero","variance",".","Defaults","to","``","1e-5","``",".","scale_init",":","Optional","initializer","for","the","scale","variable",".","Can","only","be","set","if","``","create_scale=True","``",".","By","default","scale","is","initialized","to","``","1","``",".","offset_init",":","Optional","initializer","for","the","offset","variable",".","Can","only","be","set","if","``","create_offset=True","``",".","By","default","offset","is","initialized","to","``","0","``",".","data_format",":","The","data","format","of","the","input",".","Can","be","either","``","channels_first","``",",","``","channels_last","``",",","``","N","...","C","``","or","``","NC","...","``",".","By","default","it","is","``","channels_last","``",".","name",":","Name","of","the","module","."]
code: def __init__(
      self,
      create_scale: bool,
      create_offset: bool,
      moving_mean: metrics.Metric,
      moving_variance: metrics.Metric,
      eps: types.FloatLike = 1e-5,
      scale_init: Optional[initializers.Initializer] = None,
      offset_init: Optional[initializers.Initializer] = None,
      data_format: str = "channels_last",
      name: Optional[str] = None,
  ):
      """Constructs a ``BaseBatchNorm`` module.
  
      Args:
        create_scale: whether to create a trainable scale per channel applied
          after the normalization.
        create_offset: whether to create a trainable offset per channel applied
          after normalization and scaling.
        moving_mean: A metric which tracks the moving average of the mean which
          can be used to normalize at test time.
        moving_variance: A metric which tracks the moving average of the variance
          which can be used to normalize at test time.
        eps: Small epsilon to avoid division by zero variance. Defaults to
          ``1e-5``.
        scale_init: Optional initializer for the scale variable. Can only be set
          if ``create_scale=True``. By default scale is initialized to ``1``.
        offset_init: Optional initializer for the offset variable. Can only be set
          if ``create_offset=True``. By default offset is initialized to ``0``.
        data_format: The data format of the input. Can be either
          ``channels_first``, ``channels_last``, ``N...C`` or ``NC...``. By
          default it is ``channels_last``.
        name: Name of the module.
      """
      super().__init__(name=name)
  
      self._eps = eps
  
      self.moving_mean = moving_mean
      self.moving_variance = moving_variance
  
      self._data_format = data_format
      self._channel_index = utils.get_channel_index(data_format)
  
      self._create_scale = create_scale
      self._create_offset = create_offset
  
      if not self._create_scale and scale_init is not None:
          raise ValueError(
              "Cannot set `scale_init` if `create_scale=False`"
          )
      self._scale_init = scale_init or initializers.Ones()
      if not self._create_offset and offset_init is not None:
          raise ValueError(
              "Cannot set `offset_init` if `create_offset=False`"
          )
      self._offset_init = offset_init or initializers.Zeros()
  
file_path: sonnet/sonnet/src/batch_norm.py
imports: from typing import Optional, Tuple
  from sonnet.src import base
  from sonnet.src import initializers
  from sonnet.src import metrics
  from sonnet.src import moving_averages
  from sonnet.src import once
  from sonnet.src import types
  from sonnet.src import utils
  import tensorflow
label_desc: Use image node_id 1 to create a new BaseBatchNorm object from inherited base classes: base with example: obj = BaseBatchNorm(create_scale, create_offset, moving_mean, moving_variance, eps, scale_init, offset_init, data_format, name)