Why not always use the ADAM optimization technique? It seems the Adaptive Moment Estimation (Adam) optimizer nearly always works better (faster and more reliably reaching a global minimum) when minimising the cost function in training neural nets.

Why not always use Adam? Why even bother using RMSProp or momentum optimizers?
Here’s a blog post reviewing an article claiming SGD is a better generalized adapter than ADAM.
There is often a value to using more than one method (an ensemble), because every method has a weakness.