title: How to normalize training data for different activation functions?
body: <p>I'm training fully connected neural network to classify MNIST dataset. Input data are square black-and-white images with pixel values in <code>[0, 255]</code>.</p>
  
  <p>I've read that for using <code>sigmoid()</code> activator one needs to normalize input to be in <code>[0, 1]</code> (range of sigmoid).</p>
  
  <p>How to normalize input data for <code>tanh()</code> activator? Do I need to rescale it to be in <code>[-1, 1]</code> (range of tanh) or it can still be in <code>[0, 1]</code>.</p>
  
  <p>Which approach is better and why? What is the general guidance?</p>