title: Are the time_to_batch and batch_time functions of the Tensorflow implementation of WaveNet necessary?
body: <p>Deepmind recently created a new type of convolutional architecture called WaveNet. The main innovation of WaveNet is dilated convolutions. This question is not about the modelling done, but rather about a specific programming technique they used.
  The Tensorflow implementation of WaveNet can be found <a href="https://github.com/ibab/tensorflow-wavenet" rel="nofollow noreferrer">here</a>.
  My question concerns some code found in the file 'ops.py'
  Two functions, in particular are:</p>
  
  <pre><code>def time_to_batch(value, dilation, name=None):
      with tf.name_scope('time_to_batch'):
          shape = tf.shape(value)
          pad_elements = dilation - 1 - (shape[1] + dilation - 1) % dilation
          padded = tf.pad(value, [[0, 0], [0, pad_elements], [0, 0]])
          reshaped = tf.reshape(padded, [-1, dilation, shape[2]])
          transposed = tf.transpose(reshaped, perm=[1, 0, 2])
          return tf.reshape(transposed, [shape[0] * dilation, -1, shape[2]])
  
  
  def batch_to_time(value, dilation, name=None):
      with tf.name_scope('batch_to_time'):
          shape = tf.shape(value)
          prepared = tf.reshape(value, [dilation, -1, shape[2]])
          transposed = tf.transpose(prepared, perm=[1, 0, 2])
          return tf.reshape(transposed,
                            [tf.div(shape[0], dilation), -1, shape[2]])
  </code></pre>
  
  <p>The functions are then called by some other code as in:</p>
  
  <pre><code>def causal_conv(value, filter_, dilation, name='causal_conv'):
      with tf.name_scope(name):
          filter_width = tf.shape(filter_)[0]
          if dilation &gt; 1:
              transformed = time_to_batch(value, dilation)
              conv = tf.nn.conv1d(transformed, filter_, stride=1,
                                  padding='VALID')
              restored = batch_to_time(conv, dilation)
          else:
              restored = tf.nn.conv1d(value, filter_, stride=1, padding='VALID')
          # Remove excess elements at the end.
          out_width = tf.shape(value)[1] - (filter_width - 1) * dilation
          result = tf.slice(restored,
                            [0, 0, 0],
                            [-1, out_width, -1])
          return result
  </code></pre>
  
  <p>So clearly batch_to_time and time_to_batch are only called when dilation is greater than 1. I don't quite understand how these functions work (I am still trying to learn Tensorflow) but they must squish the input down depending on the dilation, apply a regular convolution, then un-squish it back.
  However, I noticed the API provides a function called <a href="https://www.tensorflow.org/api_docs/python/tf/nn/atrous_conv2d" rel="nofollow noreferrer">atrous_conv2d</a>. An atrous convolution is another name for a dilated convolution.</p>
  
  <p>So finally, I am ready to ask my question!</p>
  
  <p>Are the aforementioned reshaping functions required, or can the dilated convolutions be achieved using atrous_conv2D with one of the dimensions set to size 1?</p>
  
  <p>After all, conv1D is implemented by calling conv2D. From <a href="https://www.tensorflow.org/api_docs/python/tf/nn/conv1d" rel="nofollow noreferrer">here</a>  :"Internally, this op reshapes the input tensors and invokes tf.nn.conv2d"</p>