class_docstr: ConvolutionTranspose layer class with Flipout estimator.
  
  This layer implements the Bayesian variational inference analogue to
  a ConvolutionTranspose layer by assuming the `kernel` and/or the `bias` are
  drawn from distributions. By default, the layer implements a stochastic
  forward pass via sampling from the kernel and bias posteriors,
  
  ```none
  kernel, bias ~ posterior
  outputs = tf.nn.conv_transpose(inputs, kernel) + bias
  ```
  
  It uses the Flipout estimator [(Wen et al., 2018)][1], which performs a Monte
  Carlo approximation of the distribution integrating over the `kernel` and
  `bias`. Flipout uses roughly twice as many floating point operations as the
  reparameterization estimator but has the advantage of significantly lower
  variance.
  
  The arguments permit separate specification of the surrogate posterior
  (`q(W|x)`), prior (`p(W)`), and divergence for both the `kernel` and `bias`
  distributions.
  
  Upon being built, this layer adds losses (accessible via the `losses`
  property) representing the divergences of `kernel` and/or `bias` surrogate
  posteriors and their respective priors. When doing minibatch stochastic
  optimization, make sure to scale this loss such that it is applied just once
  per epoch (e.g. if `kl` is the sum of `losses` for each element of the batch,
  you should pass `kl / num_examples_per_epoch` to your optimizer).
  
  ```python
  inputs = tf.transpose(inputs, tf.concat([
      [0], tf.range(2, tf.rank(inputs)), [1]], axis=0))
  ```
  
  #### Examples
  
  We illustrate a Bayesian autoencoder network with [variational inference](
  https://en.wikipedia.org/wiki/Variational_Bayesian_methods), assuming a
  dataset of images. Note that this examples is *not* a variational autoencoder,
  rather it is a Bayesian Autoencoder which also uses variational inference.
  
  ```python
  # Using the following substitution, see:
  tfn = tfp.experimental.nn
  help(tfn.ConvolutionTransposeVariationalReparameterization)
  BayesConv2D = functools.partial(
      tfn.ConvolutionVariationalFlipout,
      rank=2,
      padding='same',
      filter_shape=5,
      # Use `he_uniform` because we'll use the `relu` family.
      kernel_initializer=tf_keras.initializers.he_uniform())
  BayesDeconv2D = functools.partial(
      tfn.ConvolutionTransposeVariationalFlipout,
      rank=2,
      padding='same',
      filter_shape=5,
      # Use `he_uniform` because we'll use the `relu` family.
      kernel_initializer=tf_keras.initializers.he_uniform())
  ```
  
  This example uses reparameterization gradients to minimize the
  Kullback-Leibler divergence up to a constant, also known as the negative
  Evidence Lower Bound. It consists of the sum of two terms: the expected
  negative log-likelihood, which we approximate via Monte Carlo; and the KL
  divergence, which is added via regularizer terms which are arguments to the
  layer.
  
  #### References
  
  [1]: Yeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, and Roger Grosse. Flipout:
       Efficient Pseudo-Independent Weight Perturbations on Mini-Batches. In
       _International Conference on Learning Representations_, 2018.
       https://arxiv.org/abs/1803.04386
class_docstr_tok: ["ConvolutionTranspose","layer","class","with","Flipout","estimator",".","This","layer","implements","the","Bayesian","variational","inference","analogue","to","a","ConvolutionTranspose","layer","by","assuming","the","`","kernel","`","and\/or","the","`","bias","`","are","drawn","from","distributions",".","By","default",",","the","layer","implements","a","stochastic","forward","pass","via","sampling","from","the","kernel","and","bias","posteriors",",","``","`","none","kernel",",","bias","~","posterior","outputs","=","tf.nn.conv_transpose","(","inputs",",","kernel",")","+","bias","``","`","It","uses","the","Flipout","estimator","[","(","Wen","et","al.",",","2018",")","]","[","1","]",",","which","performs","a","Monte","Carlo","approximation","of","the","distribution","integrating","over","the","`","kernel","`","and","`","bias","`",".","Flipout","uses","roughly","twice","as","many","floating","point","operations","as","the","reparameterization","estimator","but","has","the","advantage","of","significantly","lower","variance",".","The","arguments","permit","separate","specification","of","the","surrogate","posterior","(","`","q","(","W|x",")","`",")",",","prior","(","`","p","(","W",")","`",")",",","and","divergence","for","both","the","`","kernel","`","and","`","bias","`","distributions",".","Upon","being","built",",","this","layer","adds","losses","(","accessible","via","the","`","losses","`","property",")","representing","the","divergences","of","`","kernel","`","and\/or","`","bias","`","surrogate","posteriors","and","their","respective","priors",".","When","doing","minibatch","stochastic","optimization",",","make","sure","to","scale","this","loss","such","that","it","is","applied","just","once","per","epoch","(","e.g",".","if","`","kl","`","is","the","sum","of","`","losses","`","for","each","element","of","the","batch",",","you","should","pass","`","kl","\/","num_examples_per_epoch","`","to","your","optimizer",")",".","``","`","python","inputs","=","tf.transpose","(","inputs",",","tf.concat","(","[","[","0","]",",","tf.range","(","2",",","tf.rank","(","inputs",")",")",",","[","1","]","]",",","axis=0",")",")","``","`","#","#","#","#","Examples","We","illustrate","a","Bayesian","autoencoder","network","with","[","variational","inference","]","(","https",":","\/\/en.wikipedia.org\/wiki\/Variational_Bayesian_methods",")",",","assuming","a","dataset","of","images",".","Note","that","this","examples","is","*","not","*","a","variational","autoencoder",",","rather","it","is","a","Bayesian","Autoencoder","which","also","uses","variational","inference",".","``","`","python","#","Using","the","following","substitution",",","see",":","tfn","=","tfp.experimental.nn","help","(","tfn.ConvolutionTransposeVariationalReparameterization",")","BayesConv2D","=","functools.partial","(","tfn.ConvolutionVariationalFlipout",",","rank=2",",","padding='same","'",",","filter_shape=5",",","#","Use","`","he_uniform","`","because","we","'ll","use","the","`","relu","`","family",".","kernel_initializer=tf_keras.initializers.he_uniform","(",")",")","BayesDeconv2D","=","functools.partial","(","tfn.ConvolutionTransposeVariationalFlipout",",","rank=2",",","padding='same","'",",","filter_shape=5",",","#","Use","`","he_uniform","`","because","we","'ll","use","the","`","relu","`","family",".","kernel_initializer=tf_keras.initializers.he_uniform","(",")",")","``","`","This","example","uses","reparameterization","gradients","to","minimize","the","Kullback-Leibler","divergence","up","to","a","constant",",","also","known","as","the","negative","Evidence","Lower","Bound",".","It","consists","of","the","sum","of","two","terms",":","the","expected","negative","log-likelihood",",","which","we","approximate","via","Monte","Carlo",";","and","the","KL","divergence",",","which","is","added","via","regularizer","terms","which","are","arguments","to","the","layer",".","#","#","#","#","References","[","1","]",":","Yeming","Wen",",","Paul","Vicol",",","Jimmy","Ba",",","Dustin","Tran",",","and","Roger","Grosse",".","Flipout",":","Efficient","Pseudo-Independent","Weight","Perturbations","on","Mini-Batches",".","In","_International","Conference","on","Learning","Representations_",",","2018.","https",":","\/\/arxiv.org\/abs\/1803.04386"]
docstr: Constructs layer.
  
  Note: `data_format` is not supported since all nn layers operate on
  the rightmost column. If your channel dimension is not rightmost, use
  `tf.transpose` before calling this layer. For example, if your channel
  dimension is second from the left, the following code will move it
  rightmost:
  
  ```python
  inputs = tf.transpose(inputs, tf.concat([
      [0], tf.range(2, tf.rank(inputs)), [1]], axis=0))
  ```
  
  Args:
    input_size: ...
      In Keras, this argument is inferred from the rightmost input shape,
      i.e., `tf.shape(inputs)[-1]`. This argument specifies the size of the
      second from the rightmost dimension of both `inputs` and `kernel`.
      Default value: `None`.
    output_size: ...
      In Keras, this argument is called `filters`. This argument specifies the
      rightmost dimension size of both `kernel` and `bias`.
    filter_shape: ...
      In Keras, this argument is called `kernel_size`. This argument specifies
      the leftmost `rank` dimensions' sizes of `kernel`.
    rank: An integer, the rank of the convolution, e.g. "2" for 2D
      convolution. This argument implies the number of `kernel` dimensions,
      i.e., `kernel.shape.rank == rank + 2`.
      In Keras, this argument has the same name and semantics.
      Default value: `2`.
    strides: An integer or tuple/list of n integers, specifying the stride
      length of the convolution.
      In Keras, this argument has the same name and semantics.
      Default value: `1`.
    padding: One of `"VALID"` or `"SAME"` (case-insensitive).
      In Keras, this argument has the same name and semantics (except we don't
      support `"CAUSAL"`).
      Default value: `'VALID'`.
    dilations: An integer or tuple/list of `rank` integers, specifying the
      dilation rate to use for dilated convolution. Currently, specifying any
      `dilations` value != 1 is incompatible with specifying any `strides`
      value != 1.
      In Keras, this argument is called `dilation_rate`.
      Default value: `1`.
    output_padding: An `int` or length-`rank` tuple/list representing the
      amount of padding along the input spatial dimensions (e.g., depth,
      height, width). A single `int` indicates the same value for all spatial
      dimensions. The amount of output padding along a given dimension must be
      lower than the stride along that same dimension.  If set to `None`
      (default), the output shape is inferred.
      In Keras, this argument has the same name and semantics.
      Default value: `None` (i.e., inferred).
    method: ...
    kernel_initializer: ...
      Default value: `None` (i.e.,
      `tfp.experimental.nn.initializers.glorot_uniform()`).
    bias_initializer: ...
      Default value: `None` (i.e., `tf_keras.initializers.zeros()`).
    make_posterior_fn: ...
      Default value:
        `tfp.experimental.nn.util.make_kernel_bias_posterior_mvn_diag`.
    make_prior_fn: ...
      Default value:
        `tfp.experimental.nn.util.make_kernel_bias_prior_spike_and_slab`.
    posterior_value_fn: ...
      Default valye: `tfd.Distribution.sample`
    unpack_weights_fn:
      Default value: `unpack_kernel_and_bias`
    dtype: ...
      Default value: `tf.float32`.
    index_dtype: ...
    activation_fn: ...
      Default value: `None`.
    seed: ...
      Default value: `None` (i.e., no seed).
    validate_args: ...
    name: ...
      Default value: `None` (i.e.,
      `'ConvolutionTransposeVariationalFlipout'`).
docstr_tok: ["Constructs","layer",".","Note",":","`","data_format","`","is","not","supported","since","all","nn","layers","operate","on","the","rightmost","column",".","If","your","channel","dimension","is","not","rightmost",",","use","`","tf.transpose","`","before","calling","this","layer",".","For","example",",","if","your","channel","dimension","is","second","from","the","left",",","the","following","code","will","move","it","rightmost",":","``","`","python","inputs","=","tf.transpose","(","inputs",",","tf.concat","(","[","[","0","]",",","tf.range","(","2",",","tf.rank","(","inputs",")",")",",","[","1","]","]",",","axis=0",")",")","``","`","Args",":","input_size",":","...","In","Keras",",","this","argument","is","inferred","from","the","rightmost","input","shape",",","i.e.",",","`","tf.shape","(","inputs",")","[","-1","]","`",".","This","argument","specifies","the","size","of","the","second","from","the","rightmost","dimension","of","both","`","inputs","`","and","`","kernel","`",".","Default","value",":","`","None","`",".","output_size",":","...","In","Keras",",","this","argument","is","called","`","filters","`",".","This","argument","specifies","the","rightmost","dimension","size","of","both","`","kernel","`","and","`","bias","`",".","filter_shape",":","...","In","Keras",",","this","argument","is","called","`","kernel_size","`",".","This","argument","specifies","the","leftmost","`","rank","`","dimensions","'","sizes","of","`","kernel","`",".","rank",":","An","integer",",","the","rank","of","the","convolution",",","e.g",".","``","2","''","for","2D","convolution",".","This","argument","implies","the","number","of","`","kernel","`","dimensions",",","i.e.",",","`","kernel.shape.rank","==","rank","+","2","`",".","In","Keras",",","this","argument","has","the","same","name","and","semantics",".","Default","value",":","`","2","`",".","strides",":","An","integer","or","tuple\/list","of","n","integers",",","specifying","the","stride","length","of","the","convolution",".","In","Keras",",","this","argument","has","the","same","name","and","semantics",".","Default","value",":","`","1","`",".","padding",":","One","of","`","``","VALID","''","`","or","`","``","SAME","''","`","(","case-insensitive",")",".","In","Keras",",","this","argument","has","the","same","name","and","semantics","(","except","we","don't","support","`","``","CAUSAL","''","`",")",".","Default","value",":","`","'VALID","'","`",".","dilations",":","An","integer","or","tuple\/list","of","`","rank","`","integers",",","specifying","the","dilation","rate","to","use","for","dilated","convolution",".","Currently",",","specifying","any","`","dilations","`","value","!","=","1","is","incompatible","with","specifying","any","`","strides","`","value","!","=","1",".","In","Keras",",","this","argument","is","called","`","dilation_rate","`",".","Default","value",":","`","1","`",".","output_padding",":","An","`","int","`","or","length-","`","rank","`","tuple\/list","representing","the","amount","of","padding","along","the","input","spatial","dimensions","(","e.g.",",","depth",",","height",",","width",")",".","A","single","`","int","`","indicates","the","same","value","for","all","spatial","dimensions",".","The","amount","of","output","padding","along","a","given","dimension","must","be","lower","than","the","stride","along","that","same","dimension",".","If","set","to","`","None","`","(","default",")",",","the","output","shape","is","inferred",".","In","Keras",",","this","argument","has","the","same","name","and","semantics",".","Default","value",":","`","None","`","(","i.e.",",","inferred",")",".","method",":","...","kernel_initializer",":","...","Default","value",":","`","None","`","(","i.e.",",","`","tfp.experimental.nn.initializers.glorot_uniform","(",")","`",")",".","bias_initializer",":","...","Default","value",":","`","None","`","(","i.e.",",","`","tf_keras.initializers.zeros","(",")","`",")",".","make_posterior_fn",":","...","Default","value",":","`","tfp.experimental.nn.util.make_kernel_bias_posterior_mvn_diag","`",".","make_prior_fn",":","...","Default","value",":","`","tfp.experimental.nn.util.make_kernel_bias_prior_spike_and_slab","`",".","posterior_value_fn",":","...","Default","valye",":","`","tfd.Distribution.sample","`","unpack_weights_fn",":","Default","value",":","`","unpack_kernel_and_bias","`","dtype",":","...","Default","value",":","`","tf.float32","`",".","index_dtype",":","...","activation_fn",":","...","Default","value",":","`","None","`",".","seed",":","...","Default","value",":","`","None","`","(","i.e.",",","no","seed",")",".","validate_args",":","...","name",":","...","Default","value",":","`","None","`","(","i.e.",",","`","'ConvolutionTransposeVariationalFlipout","'","`",")","."]
code: def __init__(
      self,
      input_size,
      output_size,  # keras::Conv::filters
      # ConvTranspose specific.
      filter_shape,  # keras::Conv::kernel_size
      rank=2,  # keras::Conv::rank
      strides=1,  # keras::Conv::strides
      padding="VALID",  # keras::Conv::padding; 'CAUSAL' not implemented.
      # keras::Conv::data_format is not implemented
      dilations=1,  # keras::Conv::dilation_rate
      output_padding=None,  # keras::ConvTranspose::output_padding
      method="auto",
      # Weights
      kernel_initializer=None,  # tfp.nn.initializers.glorot_uniform()
      bias_initializer=None,  # tf_keras.initializers.zeros()
      make_posterior_fn=kernel_bias_lib.make_kernel_bias_posterior_mvn_diag,
      make_prior_fn=kernel_bias_lib.make_kernel_bias_prior_spike_and_slab,
      posterior_value_fn=tfd.Distribution.sample,
      unpack_weights_fn=unpack_kernel_and_bias,
      dtype=tf.float32,
      index_dtype=tf.int32,
      # Misc
      activation_fn=None,
      seed=None,
      validate_args=False,
      name=None,
  ):
      """Constructs layer.
  
      Note: `data_format` is not supported since all nn layers operate on
      the rightmost column. If your channel dimension is not rightmost, use
      `tf.transpose` before calling this layer. For example, if your channel
      dimension is second from the left, the following code will move it
      rightmost:
  
      ```python
      inputs = tf.transpose(inputs, tf.concat([
          [0], tf.range(2, tf.rank(inputs)), [1]], axis=0))
      ```
  
      Args:
        input_size: ...
          In Keras, this argument is inferred from the rightmost input shape,
          i.e., `tf.shape(inputs)[-1]`. This argument specifies the size of the
          second from the rightmost dimension of both `inputs` and `kernel`.
          Default value: `None`.
        output_size: ...
          In Keras, this argument is called `filters`. This argument specifies the
          rightmost dimension size of both `kernel` and `bias`.
        filter_shape: ...
          In Keras, this argument is called `kernel_size`. This argument specifies
          the leftmost `rank` dimensions' sizes of `kernel`.
        rank: An integer, the rank of the convolution, e.g. "2" for 2D
          convolution. This argument implies the number of `kernel` dimensions,
          i.e., `kernel.shape.rank == rank + 2`.
          In Keras, this argument has the same name and semantics.
          Default value: `2`.
        strides: An integer or tuple/list of n integers, specifying the stride
          length of the convolution.
          In Keras, this argument has the same name and semantics.
          Default value: `1`.
        padding: One of `"VALID"` or `"SAME"` (case-insensitive).
          In Keras, this argument has the same name and semantics (except we don't
          support `"CAUSAL"`).
          Default value: `'VALID'`.
        dilations: An integer or tuple/list of `rank` integers, specifying the
          dilation rate to use for dilated convolution. Currently, specifying any
          `dilations` value != 1 is incompatible with specifying any `strides`
          value != 1.
          In Keras, this argument is called `dilation_rate`.
          Default value: `1`.
        output_padding: An `int` or length-`rank` tuple/list representing the
          amount of padding along the input spatial dimensions (e.g., depth,
          height, width). A single `int` indicates the same value for all spatial
          dimensions. The amount of output padding along a given dimension must be
          lower than the stride along that same dimension.  If set to `None`
          (default), the output shape is inferred.
          In Keras, this argument has the same name and semantics.
          Default value: `None` (i.e., inferred).
        method: ...
        kernel_initializer: ...
          Default value: `None` (i.e.,
          `tfp.experimental.nn.initializers.glorot_uniform()`).
        bias_initializer: ...
          Default value: `None` (i.e., `tf_keras.initializers.zeros()`).
        make_posterior_fn: ...
          Default value:
            `tfp.experimental.nn.util.make_kernel_bias_posterior_mvn_diag`.
        make_prior_fn: ...
          Default value:
            `tfp.experimental.nn.util.make_kernel_bias_prior_spike_and_slab`.
        posterior_value_fn: ...
          Default valye: `tfd.Distribution.sample`
        unpack_weights_fn:
          Default value: `unpack_kernel_and_bias`
        dtype: ...
          Default value: `tf.float32`.
        index_dtype: ...
        activation_fn: ...
          Default value: `None`.
        seed: ...
          Default value: `None` (i.e., no seed).
        validate_args: ...
        name: ...
          Default value: `None` (i.e.,
          `'ConvolutionTransposeVariationalFlipout'`).
      """
      filter_shape = convolution_util.prepare_tuple_argument(
          filter_shape, rank, "filter_shape", validate_args
      )
      kernel_shape = ps.convert_to_shape_tensor(
          [ps.reduce_prod(filter_shape) * input_size, output_size]
      )
      batch_ndims = 0
  
      apply_kernel_fn = _get_convolution_transpose_fn(strides, method)(
          filter_shape,
          strides,
          padding,
          rank=2,
          dilations=dilations,
          dtype=index_dtype,
          validate_args=validate_args,
      )
      super(ConvolutionTransposeVariationalFlipout, self).__init__(
          posterior=make_posterior_fn(
              kernel_shape,
              [output_size],
              kernel_initializer,
              bias_initializer,
              batch_ndims,
              batch_ndims,
              dtype,
          ),
          prior=make_prior_fn(
              kernel_shape,
              [output_size],
              kernel_initializer,
              bias_initializer,
              batch_ndims,
              batch_ndims,
              dtype,
          ),
          apply_kernel_fn=apply_kernel_fn,
          posterior_value_fn=posterior_value_fn,
          unpack_weights_fn=unpack_weights_fn,
          dtype=dtype,
          activation_fn=activation_fn,
          seed=seed,
          validate_args=validate_args,
          name=name,
      )
  
file_path: probability/tensorflow_probability/python/experimental/nn/convolutional_transpose_layers.py
imports: import tensorflow.compat.v2
  from tensorflow_probability.python.distributions import distribution
  from tensorflow_probability.python.experimental.nn import layers
  from tensorflow_probability.python.experimental.nn import variational_base
  from tensorflow_probability.python.experimental.nn.util import convolution_util
  from tensorflow_probability.python.experimental.nn.util import kernel_bias
  from tensorflow_probability.python.internal import prefer_static
label_desc: Use image node_id 1 to create a new ConvolutionTransposeVariationalFlipout object from inherited base classes: vi_lib with example: obj = ConvolutionTransposeVariationalFlipout(input_size, output_size, filter_shape, rank, strides, padding, dilations, output_padding, method, kernel_initializer, bias_initializer, make_posterior_fn, make_prior_fn, posterior_value_fn, unpack_weights_fn, dtype, index_dtype, activation_fn, seed, validate_args, name)