Why is the hyperbolic tangent with MSE better than the sigmoid with cross-entropy?
See the blog post Why You Should Use Cross-Entropy Error Instead Of Classification Error Or Mean Squared Error For Neural Network Classifier Training (2013) by James D. McCaffrey. 

It should give you an intuition of why the average cross-entropy (ACE) is more appropriate than MSE (but MSE is also applicable).

In a few words, $\tanh$ + MSE is like sigmoid + MSE, but with labels for classes $-1$ and $1$ instead of $0$ and $1$. If you look at the shape of $\tanh$ function, it has the same flat tails where changes of the argument don't change the result.