<h3 dir="auto">System information</h3>
<ul dir="auto">
<li><strong>Have I written custom code (as opposed to using a stock example script provided in TensorFlow)</strong>: No.</li>
<li><strong>OS Platform and Distribution (e.g., Linux Ubuntu 16.04)</strong>: Ubuntu 16.04</li>
<li><strong>TensorFlow installed from (source or binary)</strong>: Binary</li>
<li><strong>TensorFlow version (use command below)</strong>: 1.8.0</li>
<li><strong>Python version</strong>: 2.7.12</li>
<li><strong>Bazel version (if compiling from source)</strong>: n/a</li>
<li><strong>GCC/Compiler version (if compiling from source)</strong>: n/a</li>
<li><strong>CUDA/cuDNN version</strong>: n/a</li>
<li><strong>GPU model and memory</strong>: n/a</li>
<li><strong>Exact command to reproduce</strong>: n/a</li>
</ul>
<h3 dir="auto">Describe the problem</h3>
<p dir="auto">Currently <code class="notranslate">tf.estimator.train_and_evaluate</code> makes it easy to use an <code class="notranslate">Estimator</code> to perform both training and evaluation, possibly in a distributed environment.  However, this function only supports a single evaluation dataset.  This makes the function suboptimal because we oftentimes want to evaluate on both the training and the validation set in order to get a sense for the amount of overfitting that is happening.  It would be ideal if we could perhaps pass a list of <code class="notranslate">EvalSpec</code> objects to <code class="notranslate">train_and_evaluate</code>.</p>
<p dir="auto">Sometimes could be useful to evaluate on different clusters of the evaluation set with distinct metrics, summaries etc.<br>
Do you plan that this interface will accept a list of <code class="notranslate">eval_spec</code> in the future?</p>