Is the choice of the optimiser relevant when doing object detection?
I have experimented with this to a small degree and have not noticed that much of an impact.
To date, Adam appears to give the best results on a variety of image data sets. I have found that &quot;adjusting&quot; the learning rate during training is an effective means of improving model performance and has more impact than the selection of the optimizer.
Keras has two callbacks that are useful for this purpose. Documentation is at https://keras.io/callbacks/. The ModelCheckpoint callback enables you to save the full model or just the model weights based on monitoring a metric. Typically, you monitor validation loss and set the parameter save_best_only=True to save the results for the lowest validation loss. The other useful callback is ReduceLROnPlateau, which allows you to adjust the learning rate based on monitoring a metric. Again, the metric usually monitored is the validation loss. If the loss fails to reduce after a user-set number of epochs (parameter patience), the learning rate will be adjusted by a user-set factor (parameter factor). You can think of the training process as traveling down a valley. As you near the bottom of the valley, it becomes more and more narrow. If your learning rate does not adjust to the &quot;narrowness&quot; there is no way you will get to the bottom of the valley.
You can also write a custom callback to adjust the learning rate. I have done this and created one which first adjusts the learning rate based on monitoring the training loss until the training accuracy reaches 95%. Then it switches to adjust the learning rate based on monitoring the validation loss. It saves the model weights for the lowest validation loss and loads the model with these weights to make predictions. I have found this approach leads to faster training and higher accuracy.
The fact is you can't tell if your model has converged on a global minimum or a local minimum. This is evidenced by the fact that, unless you take special efforts to inhibit randomization, you can get different results each time you run your model. The loss can be envisioned as a surface in $N$ space, where $N$ is the number of trainable parameters. Lord knows what that surface is like and where your initial parameter weights put you on that surface, plus how other random processes cause you to traverse that surface.
As an example, I ran a model at least 20 times and got resultant losses that were very close to each there. Then I ran it again and got far better results for exactly the same data.