title: TensorFlow: Why is my Keras callback monitor value not available?
body: <p>I use TensorFlow 1.12. I try to fit a model using Keras callbacks:</p>
  
  <pre><code>checkpoint_full = tf.keras.callbacks.ModelCheckpoint(filepath='model/dir/model.{epoch:02d}.hdf5', monitor='val_dice_loss', save_best_only=True, mode="min")
  tensorboard = tf.keras.callbacks.TensorBoard(log_dir='/media/nfs/7_raid/ebos/models/fcn/logs/', write_graph=False)
  history = model.fit(train,
                      steps_per_epoch=int(np.ceil(num_samples / float(BATCH_SIZE))),
                      epochs=NUM_EPOCHS,
                      validation_data=val,
                      verbose=0,
                      callbacks=[checkpoint_full, tensorboard])
  </code></pre>
  
  <p>However, this throws the following error:</p>
  
  <pre><code>WARNING:tensorflow:Can save best model only with val_dice_loss available, skipping.
  </code></pre>
  
  <p>Firstly, I do not understand how the string given to <code>monitor</code> is being resolved. I'm following two guides on Keras (<a href="https://github.com/tensorflow/models/blob/master/samples/outreach/blogs/segmentation_blogpost/image_segmentation.ipynb" rel="nofollow noreferrer">1</a>, <a href="https://www.tensorflow.org/guide/keras" rel="nofollow noreferrer">2</a>), and both supply string names which aren't referenced anywhere else in their code. I assume these specify strings which the user can later use to retrieve the development of performance after training, rather than specify which metric <code>fit</code> is supposed to monitor? If so, why is it saying that the value isn't available?</p>
  
  <p>Secondly, answers to similar questions all point out that the problem is the absence of validation data. However, I am pretty sure that I am supplying <code>fit</code> with data, as evidenced by looping through <code>val</code> and counting the number of records. What am I doing wrong?</p>
  
  <p>For reference, my datasets are generated like this:</p>
  
  <pre><code>def train_sample_fetcher():
      return sample_fetcher()
  
  def val_sample_fetcher():
      return sample_fetcher(is_validations=True)
  
  def sample_fetcher(is_validations=False):
      sample_names = [filename[:-4] for filename in os.listdir(DIR_DATASET + "ndarrays/")]
      if not is_validations: sample_names = sample_names[:int(len(sample_names) * TRAIN_VAL_SPLIT)]
      else: sample_names = sample_names[int(len(sample_names) * TRAIN_VAL_SPLIT):]
      for sample_name in sample_names:
          rgb = tf.image.decode_jpeg(tf.read_file(DIR_DATASET + sample_name + ".jpg"))
          rgb = tf.image.resize_images(rgb, (HEIGHT, WIDTH))
          #d = tf.image.decode_jpeg(tf.read_file(DIR_DATASET + "depth/" + sample_name + ".jpg"))
          #d = tf.image.resize_images(d, (HEIGHT, WIDTH))
          #rgbd = tf.concat([rgb,d], axis=2)
          onehots = tf.convert_to_tensor(np.load(DIR_DATASET + "ndarrays/" + sample_name + ".npy"), dtype=tf.float32)
          yield rgb, onehots
  
  train = tf.data.Dataset.from_generator(generator=train_sample_fetcher, output_types=(tf.uint8, tf.uint8))
  #train = train.repeat()
  train = train.batch(BATCH_SIZE)
  #train = train.shuffle(10)
  val = tf.data.Dataset.from_generator(generator=val_sample_fetcher, output_types=(tf.uint8, tf.uint8))
  </code></pre>
  
  <p>My loss function looks as follows:</p>
  
  <pre><code>def dice_loss(y_true, y_pred):
      smooth = 1.
      y_true_f = tf.reshape(y_true, [-1]) # Flatten
      y_pred_f = tf.reshape(y_pred, [-1]) # Flatten
      intersection = tf.reduce_sum(y_true_f * y_pred_f)
      dice_coefficient = (2. * intersection + smooth) / (tf.reduce_sum(y_true_f) + tf.reduce_sum(y_pred_f) + smooth)
      loss = 1 - dice_coefficient
      return loss
  </code></pre>
  
  <p>I perform semantic segmentation, and the dice loss is calculated per pixel.</p>