title: Keras: Data augmentation affecting training speed?
body: <p>I'm currently using Tensorflow with Keras (Python3) for Deep Learning, and I'm trying to implement an efficient data generator. As Keras also offers a <a href="https://github.com/fchollet/keras/blob/master/keras/preprocessing/image.py" rel="nofollow">data generator implementation</a>, I compared the training speed with my generator.</p>
  
  <p>Using the <a href="https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py" rel="nofollow">Keras cifar10 example</a>, results showed that when using my generator the training speed is around 20% slower than when using the Keras data generator. </p>
  
  <p>So I tried playing around with the Keras generator code to see where the speedup comes from, since my generator was basically just a simple iterator returning an image batch without any preprocessing.</p>
  
  <p>I tracked it back to the function <code>random_transform</code>:</p>
  
  <pre><code>def random_transform(self, x):
    # x is a single image, so it doesn't have image number at index 0
    img_row_index = self.row_index - 1
    img_col_index = self.col_index - 1
    img_channel_index = self.channel_index - 1
  
    # use composition of homographies to generate final transform that needs to be applied
    if self.rotation_range:
      theta = np.pi / 180 * np.random.uniform(-self.rotation_range, self.rotation_range)
    else:
      theta = 0
    rotation_matrix = np.array([[np.cos(theta), -np.sin(theta), 0],
                              [np.sin(theta), np.cos(theta), 0],
                              [0, 0, 1]])
    if self.height_shift_range:
      tx = np.random.uniform(-self.height_shift_range,   self.height_shift_range) * x.shape[img_row_index]
    else:
      tx = 0
  
    if self.width_shift_range:
      ty = np.random.uniform(-self.width_shift_range,   self.width_shift_range) * x.shape[img_col_index]
    else:
      ty = 0
  
    translation_matrix = np.array([[1, 0, tx],
                                 [0, 1, ty],
                                 [0, 0, 1]])
    if self.shear_range:
      shear = np.random.uniform(-self.shear_range, self.shear_range)
    else:
      shear = 0
    shear_matrix = np.array([[1, -np.sin(shear), 0],
                           [0, np.cos(shear), 0],
                           [0, 0, 1]])
  
    if self.zoom_range[0] == 1 and self.zoom_range[1] == 1:
      zx, zy = 1, 1
    else:
      zx, zy = np.random.uniform(self.zoom_range[0], self.zoom_range[1], 2)
    zoom_matrix = np.array([[zx, 0, 0],
                          [0, zy, 0],
                          [0, 0, 1]])
  
    transform_matrix = np.dot(np.dot(np.dot(rotation_matrix, translation_matrix), shear_matrix), zoom_matrix)
  
    h, w = x.shape[img_row_index], x.shape[img_col_index]
    transform_matrix = transform_matrix_offset_center(transform_matrix, h, w)
    x = apply_transform(x, transform_matrix, img_channel_index,
                      fill_mode=self.fill_mode, cval=self.cval)
    if self.channel_shift_range != 0:
      x = random_channel_shift(x, self.channel_shift_range, img_channel_index)
  
    if self.horizontal_flip:
      if np.random.random() &lt; 0.5:
        x = flip_axis(x, img_col_index)
  
    if self.vertical_flip:
      if np.random.random() &lt; 0.5:
        x = flip_axis(x, img_row_index)
  
    # TODO:
    # channel-wise normalization
    # barrel/fisheye
    return x
  </code></pre>
  
  <p>If you remove some code in this function, the training speed decreases.</p>
  
  <p>However I am confused, as I did not know that simple preprocessing for augmentation purposes (i.e. random transformations of the original image) could affect the training speed (as in, how fast can one epoch be finished).</p>
  
  <p>Can someone please explain to me how the methods in the <code>random_transform</code> function can influence the training speed?</p>
  
  <p>To reproduce it, use the aforementioned cifar10 example and copy-paste the image generator code in this file. When removing for example the call to <code>apply_transform</code> in the function <code>random_transform</code>, the time it takes for one epoch to complete increases.</p>
  
  <p>I used:</p>
  
  <pre><code>Python 3.4
  Tensorflow 0.11.0rc1
  Keras 1.1.0
  Cuda 8.0
  1 GPU Nvidia GeForce 1070 GTX with Nvidia 367.44 Driver
  </code></pre>
  
  <p>It took me 22 seconds to complete one epoch with batch size 32 using the original code, when removing <code>apply_transform</code> (and with my own generator), the time increased to 28 seconds.</p>
  
  <p>Edit:
  Some additional behaviour I can't explain: With default parameters shear, rotation and zoom are identity matrices, so they should not have any effect on the result. As expected, the performance stays the same after removing the dot products. However, If I remove the calculations for the three matrices, the performance decreases (25 seconds instead of 22 seconds)! Even though the matrices are not used anywhere else... Is this a python / numpy code optimization bug?</p>