<p dir="auto">With this issue I would like to track the efforts in integrating the<br>
cudnn library within tensorflow.</p>
<p dir="auto">As of June 17th 2016 doing a manual grep on the repository gives these<br>
functions as being mapped from cudnn to the stream executor,</p>
<p dir="auto">From chapter 4 of the cudnn User Guide version 5.0 (April 2016):</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetVersion</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetErrorString</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnCreate</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnDestroy</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnSetStream</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetStream</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnCreateTensorDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetTensor4dDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetTensor4dDescriptorEx</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetTensor4dDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnSetTensorNdDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnDestroyTensorDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnTransformTensor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnAddTensor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnOpTensor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnSetTensor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnScaleTensor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnCreateFilterDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetFilter4dDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetFilter4dDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetFilter4dDescriptor_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetFilter4dDescriptor_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetFilter4dDescriptor_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetFilter4dDescriptor_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnSetFilterNdDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetFilterNdDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetFilterNdDescriptor_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetFilterNdDescriptor_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetFilterNdDescriptor_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetFilterNdDescriptor_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnDestroyFilterDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnCreateConvolutionDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetConvolution2dDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetConvolution2dDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetConvolution2dForwardOutputDim</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnSetConvolutionNdDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetConvolutionNdDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetConvolutionNdForwardOutputDim</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnDestroyConvolutionDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnFindConvolutionForwardAlgorithm</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnFindConvolutionForwardAlgorithmEx</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetConvolutionForwardAlgorithm</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetConvolutionForwardWorkspaceSize</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnConvolutionForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnConvolutionBackwardBias</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnFindConvolutionBackwardFilterAlgorithm</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnFindConvolutionBackwardFilterAlgorithmEx</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetConvolutionBackwardFilterAlgorithm</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetConvolutionBackwardFilterWorkspaceSize</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnConvolutionBackwardFilter</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnFindConvolutionBackwardDataAlgorithm</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnFindConvolutionBackwardDataAlgorithmEx</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetConvolutionBackwardDataAlgorithm</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetConvolutionBackwardDataWorkspaceSize</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnConvolutionBackwardData</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSoftmaxForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSoftmaxBackward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnCreatePoolingDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetPooling2dDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetPooling2dDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnSetPoolingNdDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetPoolingNdDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetPooling2dDescriptor_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetPooling2dDescriptor_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetPoolingNdDescriptor_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetPoolingNdDescriptor_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetPooling2dDescriptor_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetPooling2dDescriptor_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetPoolingNdDescriptor_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetPoolingNdDescriptor_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnDestroyPoolingDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetPooling2dForwardOutputDim</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetPoolingNdForwardOutputDim</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnPoolingForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnPoolingBackward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnActivationForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnActivationBackward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnCreateActivationDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnSetActivationDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnGetActivationDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> cudnnDestroyActivationDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnActivationForward_v3  (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnActivationBackward_v3 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnActivationForward_v4  (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnActivationBackward_v4 (versioned)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnCreateLRNDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetLRNDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetLRNDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDestroyLRNDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnLRNCrossChannelForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnLRNCrossChannelBackward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDivisiveNormalizationForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDivisiveNormalizationBackward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnBatchNormalizationForwardInference</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnBatchNormalizationForwardTraining</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnBatchNormalizationBackward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDeriveBNTensorDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnCreateRNNDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDestroyRNNDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetRNNDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetRNNWorkspaceSize</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetRNNTrainingReserveSize</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetRNNParamsSize</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetRNNLinLayerMatrixParams</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnGetRNNLinLayerBiasParams</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnRNNForwardInference</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnRNNForwardTraining</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnRNNBackwardData</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnRNNBackwardWeights</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnCreateDropoutDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDestroyDropoutDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDropoutGetStatesSize</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDropoutGetReserveSpaceSize</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetDropoutDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDropoutForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDropoutBackward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnCreateSpatialTransformerDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnDestroySpatialTransformerDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSetSpatialTransformerNdDescriptor</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSpatialTfGridGeneratorForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSpatialTfGridGeneratorBackward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSpatialTfSamplerForward</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> cudnnSpatialTfSamplerBackward</li>
</ul>
<h3 dir="auto">Batch Normalization</h3>
<p dir="auto">Seems <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/lukemetz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/lukemetz">@lukemetz</a> was working on it but has stalled for a bit <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="145555764" data-permission-text="Title is private" data-url="https://github.com/tensorflow/tensorflow/issues/1759" data-hovercard-type="pull_request" data-hovercard-url="/tensorflow/tensorflow/pull/1759/hovercard" href="https://github.com/tensorflow/tensorflow/pull/1759">#1759</a></p>
<h3 dir="auto">What is the plan for the RNN ?</h3>
<p dir="auto">I know <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/wchan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/wchan">@wchan</a> was working on a <a href="https://github.com/tensorflow/tensorflow/pull/2002" data-hovercard-type="pull_request" data-hovercard-url="/tensorflow/tensorflow/pull/2002/hovercard">cpu version</a>. I was trying to get a stub at <a href="https://github.com/Mistobaan/tensorflow/tree/feature/cudnn-rnn-lstm">using the cudnn version</a> but from the comments in that thread seemed like <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zheng-xq/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zheng-xq">@zheng-xq</a> was working on it internally.<br>
Can anyone comment on the status of these ops?</p>
<h3 dir="auto">Other Questions</h3>
<ol dir="auto">
<li>Any reasons why the Softmax functions are not being used ?</li>
<li>Would make sense to split the above list in chunks and create issues with contribution welcome, so external contributors can tackle them without duplicating internal work, or mark what google will be working on internally ?</li>
</ol>
<p dir="auto">I hope this helps in organizing the work around the cudnn and inspire the community to contribute. I will try to keep this issue up to date.</p>
<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac OS</li>
<li>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:</li>
<li>TensorFlow installed from (source or binary): Yes</li>
<li>TensorFlow version (use command below):</li>
<li>Python version: 3.5</li>
<li>Bazel version (if compiling from source):</li>
<li>GCC/Compiler version (if compiling from source):</li>
<li>CUDA/cuDNN version:</li>
<li>GPU model and memory:</li>
</ul>
<p dir="auto"><strong>Describe the current behavior</strong></p>
<p dir="auto">when using a keras model and transforming it with <code class="notranslate">model_to_estimator</code> accuracy and loss are not stored in<code class="notranslate">events.out.tfevents.xxxxxxxxxx</code> (used by TensorBoard for visualiztion). I am using <code class="notranslate">tf.estimator.train()</code> or <code class="notranslate">tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)</code> and in both cases I only get:<br>
<code class="notranslate">dict_keys(['global_step/sec', 'loss_1'])</code></p>
<p dir="auto">using <code class="notranslate">tf.estimator.train()</code> I am expecting loss and accuray in <code class="notranslate">events.out.tfevents.xxxxxxxxxx</code> for the training dataset</p>
<p dir="auto">using <code class="notranslate">tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)</code> I am expecting loss and accuray in <code class="notranslate">events.out.tfevents.xxxxxxxxxx</code> for the training dataset abd testing datset.</p>
<p dir="auto">The info are computed and display in the logfiles:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [10/100]
INFO:tensorflow:Evaluation [20/100]
INFO:tensorflow:Evaluation [30/100]
INFO:tensorflow:Evaluation [40/100]
INFO:tensorflow:Evaluation [50/100]
INFO:tensorflow:Evaluation [60/100]
INFO:tensorflow:Evaluation [70/100]
INFO:tensorflow:Finished evaluation at 2019-03-16-12:28:05
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.9766614, global_step = 1000, loss = 0.074898034
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: results/Models/Mnist/tf_1_12/estimator/ckpt/model.ckpt-1000"><pre class="notranslate"><code class="notranslate">INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [10/100]
INFO:tensorflow:Evaluation [20/100]
INFO:tensorflow:Evaluation [30/100]
INFO:tensorflow:Evaluation [40/100]
INFO:tensorflow:Evaluation [50/100]
INFO:tensorflow:Evaluation [60/100]
INFO:tensorflow:Evaluation [70/100]
INFO:tensorflow:Finished evaluation at 2019-03-16-12:28:05
INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.9766614, global_step = 1000, loss = 0.074898034
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1000: results/Models/Mnist/tf_1_12/estimator/ckpt/model.ckpt-1000
</code></pre></div>
<p dir="auto">Is this expected ? It is documented somehwere ? without such info in TensorBoard then it is impossible to see if the model is overfitting or not. I am not even sure which info is displayed in the logfile, right ? or is there some other magic way to get such info ?</p>
<p dir="auto">why tf.summary information are not propagated from keras to estimator model ? We don't have much control over the estimator model when it is converted from Keras</p>
<p dir="auto">One option is to use a estimator model using keras layer so we can control everything but since Keras is now the official high level API from TensorFlow, I was thinking I could just transform it to an estimator model. I will be happy to test TF 2.0 if this will fix my issue.</p>
<p dir="auto"><strong>Describe the expected behavior</strong></p>
<p dir="auto">when I am using the same keras model and running the <code class="notranslate">fit</code> method, I got in the <code class="notranslate">events.out.tfevents.xxxxxxxxxx</code> file:</p>
<p dir="auto"><code class="notranslate">dict_keys(['batch_acc', 'batch_loss', 'epoch_acc', 'epoch_loss', 'epoch_val_acc', 'epoch_val_loss'])</code></p>
<p dir="auto">I got 6 variables inluding accuracy and loss for the training and test datset. This is what Iam expected.</p>
<p dir="auto"><strong>Code to reproduce the issue</strong><br>
code to inspect <code class="notranslate">events.out.tfevents.xxxxxxxxxx</code> files:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="from tensorboard.backend.event_processing import event_accumulator
import numpy as np

def load_data_tensorboard(path):
    event_acc = event_accumulator.EventAccumulator(path)
    event_acc.Reload()
    data = {}
    
    for tag in sorted(event_acc.Tags()[&quot;scalars&quot;]):
        x, y = [], []
        for scalar_event in event_acc.Scalars(tag):
            x.append(scalar_event.step)
            y.append(scalar_event.value)
        data[tag] = (np.asarray(x), np.asarray(y))
    return data"><pre class="notranslate"><code class="notranslate">from tensorboard.backend.event_processing import event_accumulator
import numpy as np

def load_data_tensorboard(path):
    event_acc = event_accumulator.EventAccumulator(path)
    event_acc.Reload()
    data = {}
    
    for tag in sorted(event_acc.Tags()["scalars"]):
        x, y = [], []
        for scalar_event in event_acc.Scalars(tag):
            x.append(scalar_event.step)
            y.append(scalar_event.value)
        data[tag] = (np.asarray(x), np.asarray(y))
    return data
</code></pre></div>
<p dir="auto">one notebook contain the code but this is "work in progress":<br>
<a href="https://github.com/tarrade/proj_DL_models_and_pipelines_with_GCP/blob/master/notebook/08-Mnist_keras_estimator.ipynb">https://github.com/tarrade/proj_DL_models_and_pipelines_with_GCP/blob/master/notebook/08-Mnist_keras_estimator.ipynb</a></p>