title: what raises StopIteration in mine Keras Model.fit_generator
body: <p>I have next code:</p>
  
  <pre><code>from sklearn.model_selection import train_test_split
  from scipy.misc import imresize
  
  def _chunks(l, n):
      """Yield successive n-sized chunks from l."""
      for i in range(0, len(l), n):
          yield l[i:i + n]
  
  
  def _batch_generator(data, batch_size):
      indexes = range(len(data))
      index_chunks = _chunks(indexes, batch_size)
      for i, indexes in enumerate(index_chunks):
          print("\nLoaded batch {0}\n".format(i + 1))
          batch_X = []
          batch_y = []
          for index in indexes:
              record = data[index]
              image = _read_train_image(record["id"], record["index"])
              mask = _read_train_mask(record["id"], record["index"])
              mask_resized = imresize(mask, (1276, 1916)) &gt;= 123
              mask_reshaped = mask_resized.reshape((1276, 1916, 1))
              batch_X.append(image)
              batch_y.append(mask_reshaped)
          np_batch_X = np.array(batch_X)
          np_batch_y = np.array(batch_y)
          yield np_batch_X, np_batch_y
  
  
  def train(data, model, batch_size, epochs):
      train_data, test_data = train_test_split(data)
      samples_per_epoch = len(train_data)
      steps_per_epoch = samples_per_epoch // batch_size
      print("Train on {0} records ({1} batches)".format(samples_per_epoch, steps_per_epoch))
      train_generator = _batch_generator(train_data, batch_size)
      model.fit_generator(train_generator, 
                          steps_per_epoch=steps_per_epoch, 
                          nb_epoch=epochs, 
                          verbose=1)
  
  train(train_indexes[:30], autoencoder,
      batch_size=2,
      epochs=1)
  </code></pre>
  
  <p>So seems like it must works next way:</p>
  
  <ul>
  <li>get 30 (just example) indexes from dataset</li>
  <li>spit it to 22 train records and 8 validate indexes (not used yet)</li>
  <li>split train indexes to batches of 2 index in generator (so - 11 batches) and it's works - <code>len(list(_batch_generator(train_indexes[:22], 2)))</code> really returns 11</li>
  <li>fit model:
  
  <ul>
  <li>on batches generated by train_generator (in mine case - 11 batches, each - 2 images)</li>
  <li>with 11 batches in epoch (<code>steps_per_epoch=steps_per_epoch</code>)</li>
  <li>and 1 epoch (<code>nb_epochs=epochs</code>, <code>epochs=1</code>)</li>
  </ul></li>
  </ul>
  
  <p>But output has next view:</p>
  
  <pre><code>Train on 22 records (11 batches)
  Epoch 1/1
  
  Loaded batch 1
  
  C:\Users\user\venv\machinelearning\lib\site-packages\ipykernel_launcher.py:39: UserWarning: The semantics of the Keras 2 argument `steps_per_epoch` is not the same as the Keras 1 argument `samples_per_epoch`. `steps_per_epoch` is the number of batches to draw from the generator at each epoch. Basically steps_per_epoch = samples_per_epoch/batch_size. Similarly `nb_val_samples`-&gt;`validation_steps` and `val_samples`-&gt;`steps` arguments have changed. Update your method calls accordingly.
  C:\Users\user\venv\machinelearning\lib\site-packages\ipykernel_launcher.py:39: UserWarning: Update your `fit_generator` call to the Keras 2 API: `fit_generator(&lt;generator..., steps_per_epoch=11, verbose=1, epochs=1)`
  
  Loaded batch 2
  
  1/11 [=&gt;............................] - ETA: 11s - loss: 0.7471
  Loaded batch 3
  
  
  Loaded batch 4
  
  
  Loaded batch 5
  
  
  Loaded batch 6
  
  2/11 [====&gt;.........................] - ETA: 17s - loss: 0.7116
  Loaded batch 7
  
  
  Loaded batch 8
  
  
  Loaded batch 9
  
  
  Loaded batch 10
  
  3/11 [=======&gt;......................] - ETA: 18s - loss: 0.6931
  Loaded batch 11
  
  Exception in thread Thread-50:
  Traceback (most recent call last):
  File "C:\Anaconda3\Lib\threading.py", line 916, in _bootstrap_inner
      self.run()
  File "C:\Anaconda3\Lib\threading.py", line 864, in run
      self._target(*self._args, **self._kwargs)
  File "C:\Users\user\venv\machinelearning\lib\site-packages\keras\utils\data_utils.py", line 560, in data_generator_task
      generator_output = next(self._generator)
  StopIteration
  
  4/11 [=========&gt;....................] - ETA: 18s - loss: 0.6663
  ---------------------------------------------------------------------------
  StopIteration                             Traceback (most recent call last)
  &lt;ipython-input-16-092ba6eb51d2&gt; in &lt;module&gt;()
      1 train(train_indexes[:30], autoencoder,
      2       batch_size=2,
  ----&gt; 3       epochs=1)
  
  &lt;ipython-input-15-f2fec4e53382&gt; in train(data, model, batch_size, epochs)
      37                         steps_per_epoch=steps_per_epoch,
      38                         nb_epoch=epochs,
  ---&gt; 39                         verbose=1)
  
  C:\Users\user\venv\machinelearning\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
      85                 warnings.warn('Update your `' + object_name +
      86                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
  ---&gt; 87             return func(*args, **kwargs)
      88         wrapper._original_function = func
      89         return wrapper
  
  C:\Users\user\venv\machinelearning\lib\site-packages\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, initial_epoch)
  1807                 batch_index = 0
  1808                 while steps_done &lt; steps_per_epoch:
  -&gt; 1809                     generator_output = next(output_generator)
  1810 
  1811                     if not hasattr(generator_output, '__len__'):
  
  StopIteration: 
  </code></pre>
  
  <p>So as I can see - all batches are readed successfylly (see "Loaded batch")</p>
  
  <p>But StopIteration is raised by keras during processing batch 3 of epoch 1.</p>