title: Using the same TPU model for training and inference (prediction) in Google Colab
body: <p>I have a code something like this:</p>
  
  <pre><code>def getModel():
      model = Sequential()
      model.Add(...)
      .....
      model = tf.contrib.tpu.keras_to_tpu_model(model,
              strategy=tf.contrib.tpu.TPUDistributionStrategy(
              tf.contrib.cluster_resolver.TPUClusterResolver(tpu='grpc://' + os.environ['COLAB_TPU_ADDR'])
          ))
      model.compile(loss='mse',
                    optimizer=tf.train.AdamOptimizer(learning_rate=1e-3, ))
      return model
  
  tpu_model = getModel()
  ## Main loop
      ....
      tpu_model.predict(states)
      tpu_model.fit(...)
  </code></pre>
  
  <p>Note that I use the same <code>tpu_model</code> for batch prediction and training.  </p>
  
  <p><code>tpu_model.predict()</code> seems to work fine, but when it runs <code>tpu_model.fit(...)</code>, it throws the following error:</p>
  
  <pre><code>WARNING:tensorflow:tpu_model (from tensorflow.contrib.tpu.python.tpu.keras_support) is experimental and may change or be removed at any time, and without warning.
  INFO:tensorflow:New input shapes; (re-)compiling: mode=infer (# of cores 8), [TensorSpec(shape=(4, 7), dtype=tf.float32, name='dense_6_input_10')]
  INFO:tensorflow:Overriding default placeholder.
  INFO:tensorflow:Remapping placeholder for dense_6_input
  INFO:tensorflow:Started compiling
  INFO:tensorflow:Finished compiling. Time elapsed: 1.464857578277588 secs
  INFO:tensorflow:Setting weights on TPU model.
  ...
  ...
  ...
  RuntimeError                              Traceback (most recent call last)
  --&gt; 101         history = tpu_model.fit(states, target_f, epochs=1, verbose=0)
      102         # Keeping track of loss
      103         loss = history.history['loss'][0]
  
  /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/tpu/python/tpu/keras_support.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
     1505                                   validation_split, validation_data, shuffle,
     1506                                   class_weight, sample_weight, initial_epoch,
  -&gt; 1507                                   steps_per_epoch, validation_steps, **kwargs)
     1508       finally:
     1509         self._numpy_to_infeed_manager_list = []
  
  /usr/local/lib/python3.6/dist-packages/tensorflow/contrib/tpu/python/tpu/keras_support.py in _pipeline_fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)
     1578         steps_name='steps_per_epoch',
     1579         steps=steps_per_epoch,
  -&gt; 1580         validation_split=validation_split)
     1581 
     1582     # Prepare validation data
  
  /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, batch_size, check_steps, steps_name, steps, validation_split)
      990         x, y, sample_weight = next_element
      991     x, y, sample_weights = self._standardize_weights(x, y, sample_weight,
  --&gt; 992                                                      class_weight, batch_size)
      993     return x, y, sample_weights
      994 
  
  /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in _standardize_weights(self, x, y, sample_weight, class_weight, batch_size)
     1036     if y is not None:
     1037       if not self.optimizer:
  -&gt; 1038         raise RuntimeError('You must compile a model before '
     1039                            'training/testing. '
     1040                            'Use `model.compile(optimizer, loss)`.')
  
  RuntimeError: You must compile a model before training/testing. Use `model.compile(optimizer, loss)`.
  </code></pre>
  
  <p>As you can see from the logs, there seems to be two modes to run on TPU:<br>
  1. <code>mode=infer</code><br>
  2. <code>mode=training</code></p>
  
  <p>It seems the both cannot be done simultaneously. Is there any way around this?</p>
  
  <p>I cannot use a Generator because I'm doing Reinforcement Learning where the batch is based on live samples added to a list dynamically from which a batch is sampled, predicted (and certain values are changed) and trained.</p>