Non differentiable loss function
You can optimize with non-gradient based methods. The field is called derivative-free optimization.
Local Search is one common approach for derivative-free optimization of neural networks.