Two common activation functions used in deep learning are the hyperbolic tangent function and the sigmoid activation function. I understand that the hyperbolic tangent is just a rescaling and translation of the sigmoid function (i,e tanh(z) = 2*sigma(z) - 1). Is there a significant difference between these two activation functions, and in particular, when is one preferable to the other?I realize that in some cases (like when estimating probabilities) outputs in the range of [0,1] are more convenient than outputs that range from [-1,1]. I want to know if there are differences other than convenience which distinguish the two activation functions.
What's the difference between hyperbolic tangent and sigmoid neurons?
deep network;neural networks;machine learning;hidden layers