A practical reason to use Cross-entropy as a error-function in Neural networks?
During back-propagation training, you want to drive output node values to either 1.0 or 0.0 depending on the target values. 
If you use MSE, the weight adjustment factor (the gradient) contains a term of (output) * (1 – output). As the computed output gets closer and closer to either 0.0 or 1.0 the value of (output) * (1 – output) gets smaller and smaller. 
For example, if output = 0.6 then (output) * (1 – output) = 0.24 but if output is 0.95 then (output) * (1 – output) = 0.0475. 
As the adjustment factor gets smaller and smaller, the change in weights gets smaller and smaller and training can stall out, so to speak.
But if you use cross-entropy error, the (output) * (1 – output) term goes away (the math is very cool). So, the weight changes don’t get smaller and smaller and so training isn’t s likely to stall out. 


Note that this argument assumes you’re doing neural network classification, with either softmax output node activation plus multiclass logloss, or sigmoid output node activation plus binary  logloss.

Reference: https://jamesmccaffrey.wordpress.com/2013/11/05/why-you-should-use-cross-entropy-error-instead-of-classification-error-or-mean-squared-error-for-neural-network-classifier-training/