Should activation function be monotonic in neural networks?
During the training phase, backpropagation informs each neuron how much it should influence each neuron in the next layer.  If the activation function isn't monotonic then increasing the neuron's weight might cause it to have less influence, the opposite of what was intended.  The result would be choatic behavior during training, with the network unlikely to converge to a state that yields an accurate classifier.