Why do activation functions need to be differentiable in the context of neural networks?
It is almost mandatory to have a differentiable activation function unless, of course, you have an alternative to training the network by back-propagating the error.