title: Tensorflow Estimator - warm_start_from and model_dir
body: <p>When using <code>tf.estimator</code> with <code>warm_start_from</code> <strong>and</strong> <code>model_dir</code>, and both <code>warm_start_from</code> directory and <code>model_dir</code> directory contain valid checkpoints, which checkpoint will be actually restored?</p>
  
  <p>To give some context, my estimator code looks like</p>
  
  <pre><code>est = tf.estimator.Estimator(
      model_fn=model_fn,
      model_dir=model_dir,
      warm_start_from=warm_start_dir)
  
  for epoch in range(num_epochs):
      est.train(input_fn=train_input_fn)
      est.evaluate(input_fn=eval_input_fn)
  </code></pre>
  
  <p>(Input functions use one shot iterators.)</p>
  
  <p>So during the first iteration, when <code>model_dir</code> is empty, I want the warm start checkpoint to be loaded, but in the next epoch, i'd like to have the intermediate fine-tuned checkpoint from the last iteration in <code>model_dir</code> to be loaded. But at least from the logs, it looks like <code>warm_start_dir</code> is still being loaded.</p>
  
  <p>I could probably override my estimator for the next iterations but I wonder if it shouldn't be built in the estimator some how.</p>