What should I observe when choosing which optimizer suits my Deep Neural Network model?
High training score is not an indication of model performance, high test score is. Also, a faster convergence to the same, or better test score is an indication of optimizer performance. 

Therefore, if Momentum optimizer reaches a better test score faster, it definitely means that it is the best out of those tested optimizers.

As a side note, be careful about the choice of "score", for example using "accuracy" for imbalanced classes is not a good choice, since it equates %1 error in a class with 100 members to 10% error in a class with 10 members. If classes are equally important, i.e. 1% error is equally bad for all classes, macro-f1 and AUC would be better replacements.

An important note: when we use a score to select a hyper-parameter or an optimizer, final model will be affected, thus, that score would be called validation score, not test score.