title: Is there a tensorflow way to extract/save mean and std used for normalization?
body: <p>I am normalizing my input training data using <code>data_norm = tf.nn.l2_normalize(data, 0)</code>.</p>
  
  <p>The data is of shape <code>[None, 4]</code>. Each column is a feature. It might look like this:</p>
  
  <pre><code>data = [[-3., 0.2, 1.6, 0.5], 
          [3.6, 1.5, -1.9, 0.71], 
         ...]
  </code></pre>
  
  <p>I understand that given normalization in the training set, the <strong>test set</strong> should be normalized, too, but using the <code>std</code> and <code>mean</code> from the <strong>training set</strong>. (I assume this applies also during the actual usage of the NN, i.e. input should be normalized using the training set <code>mean</code>, <code>std</code> before feeding into the NN).</p>
  
  <p>Is there a way to extract/save the <code>mean</code>, <code>std</code> used for normalization from this function, so I can normalize my test set using the same mean, std used for normalizing the training data?
  I know how to save the weights etc with <code>saver.save(sess, "checkpoints/" + save_id)</code> . Is there a way to save/load std, mean like this?</p>