title: Why does normalizing MNIST images reduce accuracy?
body: <p>I am using a basic NN to train and test accuracy on MNIST dataset. </p>
  
  <p>System: i5 8th Gen , GPU - Nvidia 1050Ti</p>
  
  <p>Here is my code: </p>
  
  <pre><code>from __future__ import print_function,absolute_import,unicode_literals,division
  import tensorflow as tf
  
  mnist = tf.keras.datasets.mnist
  
  (x_train,y_train) , (x_test,y_test) = mnist.load_data()
  #x_train , y_train = x_train/255.0 , y_train/255.0
  
  model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28,28)),
  tf.keras.layers.Dense(312,activation='relu'),
  tf.keras.layers.Dropout(0.3),
  tf.keras.layers.Dense(10,activation='softmax')
  ])
  
  model.compile(
  optimizer='Adamax',
  loss="sparse_categorical_crossentropy",
  metrics=['accuracy']
  )
  model.fit(x_train,y_train,epochs=5)
  model.evaluate(x_test,y_test)
  </code></pre>
  
  <p>When i normalize the images as in the 5th line, the accuracy drops horribly :</p>
  
  <pre><code>loss: 10392.0626 - accuracy: 0.0980
  </code></pre>
  
  <p>However when i dont normalize them, It gives :</p>
  
  <pre><code>- loss: 0.2409 - accuracy: 0.9420
  </code></pre>
  
  <p>In general , normalizing the data helps the grad descent to converge faster. Why is this huge difference?
  What am i missing?</p>