Keras "Tanh Activation" function -- edit: hidden layers
First of all you simply should'nt use them in your output layer. Depending on your loss function you may even get an error. A loss function like mse should be able to take the ouput of tanh, but it won't make much sense.
But if were talking about hidden layers you're perfectly fine. Also keep in mind, that there are biases which can train an offset to the layer before giving the ouput of the layer to the activation function.
Tanh activation functions bounds the output to [-1,1]. I wonder how does it work, if the input (features & Target Class) is given in 1-hot-Encoded form  ?
How keras (is managing internally) the negative output of activation function to compare them with the class labels (which are in one-hot-encoded form) -- means only 0's and 1's (no "-"ive values)
Thanks!