class_docstr: Layer wrapper to decouple magnitude and direction of the layer's weights.
  
  This wrapper reparameterizes a layer by decoupling the weight's
  magnitude and direction. This speeds up convergence by improving the
  conditioning of the optimization problem. It has an optional data-dependent
  initialization scheme, in which initial values of weights are set as functions
  of the first minibatch of data. Both the weight normalization and data-
  dependent initialization are described in [Salimans and Kingma (2016)][1].
  
  #### Example
  
  ```python
    net = WeightNorm(tf_keras.layers.Conv2D(2, 2, activation='relu'),
           input_shape=(32, 32, 3), data_init=True)(x)
    net = WeightNorm(tf_keras.layers.Conv2DTranspose(16, 5, activation='relu'),
                     data_init=True)
    net = WeightNorm(tf_keras.layers.Dense(120, activation='relu'),
                     data_init=True)(net)
    net = WeightNorm(tf_keras.layers.Dense(num_classes),
                     data_init=True)(net)
  ```
  
  #### References
  
  [1]: Tim Salimans and Diederik P. Kingma. Weight Normalization: A Simple
       Reparameterization to Accelerate Training of Deep Neural Networks. In
       _30th Conference on Neural Information Processing Systems_, 2016.
       https://arxiv.org/abs/1602.07868
class_docstr_tok: ["Layer","wrapper","to","decouple","magnitude","and","direction","of","the","layer","'s","weights",".","This","wrapper","reparameterizes","a","layer","by","decoupling","the","weight's","magnitude","and","direction",".","This","speeds","up","convergence","by","improving","the","conditioning","of","the","optimization","problem",".","It","has","an","optional","data-dependent","initialization","scheme",",","in","which","initial","values","of","weights","are","set","as","functions","of","the","first","minibatch","of","data",".","Both","the","weight","normalization","and","data-","dependent","initialization","are","described","in","[","Salimans","and","Kingma","(","2016",")","]","[","1","]",".","#","#","#","#","Example","``","`","python","net","=","WeightNorm","(","tf_keras.layers.Conv2D","(","2",",","2",",","activation='relu","'",")",",","input_shape=","(","32",",","32",",","3",")",",","data_init=True",")","(","x",")","net","=","WeightNorm","(","tf_keras.layers.Conv2DTranspose","(","16",",","5",",","activation='relu","'",")",",","data_init=True",")","net","=","WeightNorm","(","tf_keras.layers.Dense","(","120",",","activation='relu","'",")",",","data_init=True",")","(","net",")","net","=","WeightNorm","(","tf_keras.layers.Dense","(","num_classes",")",",","data_init=True",")","(","net",")","``","`","#","#","#","#","References","[","1","]",":","Tim","Salimans","and","Diederik","P.","Kingma",".","Weight","Normalization",":","A","Simple","Reparameterization","to","Accelerate","Training","of","Deep","Neural","Networks",".","In","_30th","Conference","on","Neural","Information","Processing","Systems_",",","2016.","https",":","\/\/arxiv.org\/abs\/1602.07868"]
docstr: Data dependent initialization.
docstr_tok: ["Data","dependent","initialization","."]
code: def _data_dep_init(self, inputs):
      """Data dependent initialization."""
      # Normalize kernel first so that calling the layer calculates
      # `tf.dot(v, x)/tf.norm(v)` as in (5) in ([Salimans and Kingma, 2016][1]).
      self._compute_weights()
  
      activation = self.layer.activation
      self.layer.activation = None
  
      use_bias = self.layer.bias is not None
      if use_bias:
          bias = self.layer.bias
          self.layer.bias = tf.zeros_like(bias)
  
      # Since the bias is initialized as zero, setting the activation to zero and
      # calling the initialized layer (with normalized kernel) yields the correct
      # computation ((5) in Salimans and Kingma (2016))
      x_init = self.layer(inputs)
      norm_axes_out = list(range(x_init.shape.rank - 1))
      m_init, v_init = tf.nn.moments(x_init, norm_axes_out)
      scale_init = 1.0 / tf.sqrt(v_init + 1e-10)
  
      self.g.assign(self.g * scale_init)
      if use_bias:
          self.layer.bias = bias
          self.layer.bias.assign(-m_init * scale_init)
      self.layer.activation = activation
  
file_path: probability/tensorflow_probability/python/layers/weight_norm.py
imports: import warnings
  import tensorflow.compat.v2
  from tensorflow_probability.python.internal import tf_keras
label_desc: Use image node_id 4 for calling the WeightNorm obj's underlying member method code with example usage: obj._data_dep_init(inputs) without return types