On which data evaluate an object detection model ? (similar or real life data ?) I'm training an object detection model (SSD300) to detect and classify body poses in thermal images.

Even I have more than 2k different poses, but the background does not change much (I have only 5 different points of view).

I trained my model on these images (70% for the training and 30% for validation).

Now, I want to evaluate the model on an unbiased dataset.

Should I keep images of my dataset for this purpose or should I use a real life dataset ?

(A good solution would be to have a real life training set, but I don't have)

I tried both, but as expected, I have an mAP=0.9 when evaluated on similar pictures and mAP=0.5 when evaluated on completely different images.

Bonus question: is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him "my model has a mAP=0.7")

Precision-Recall ? (but I have to choose a pose classification threshold...)
If you want to evaluate on real thermal image dataset, you can use this one.

Thermal Image dataset

is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him "my model has a mAP=0.7")


  Mean Average Precision is the relevant metric but it's more technical. You can start explaining with False Positives and False 
  Negatives in the predictions. In turn, it leads to Precision and Recall. It mostly depends upon your use case because there will always be a trade-off between them.