Choosing between two object detection model checkpoints I have the mAP scores for an object detection model evaluated at different model checkpoints. I want to choose the model that has the maximum recall on the test dataset, meaning the model which missed the least number of objects. However, the model that achieves this doesn't have the highest mAP.

mAP is the most common metric for evaluating an object detection model. How do you interpret this metric?

For the best real world performance, is it better to:


Choose a model checkpoint based on the least number of false negatives, even though the mAP isn't the highest?
Choose a model checkpoint with the highest mAP, even if it means the number of false negatives is higher?
Choosing the model with the lowest loss on the validation set would be more appropriate. There are cases where the lowest loss model also exhibits the highest mAP, but that definitely is not the norm. 

Regarding your question, if you are to choose the model based on accuracy-like metrics, consider what would be less harmful in your specific example. For instance, consider the case of cancer classification, predicting that a healthy patient has cancer (False Positive) might result in some frustration but it is not that big of a deal. On the other hand, predicting a cancer patient as of being healthy might cost his life.