On which data evaluate an object detection model ? (similar or real life data ?) I'm training an object detection model (SSD300) to detect and classify body poses in thermal images.

Even I have more than 2k different poses, the background does not change much (I have only 5 different points of view).

I trained my model on these images (70% for the training and 30% for validation).

Now, I want to evaluate the model on an unbiased dataset.

Should I keep images of my dataset for this purpose or should I use a real life dataset ?

(A good solution would be to have a real life training set, but I don't have)

I tried both, but as expected, I have an mAP=0.9 when evaluated on similar pictures and mAP=0.5 when evaluated on completely different images.

Bonus question: is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him "my model has a mAP=0.7")

Precision-Recall ? (but I have to choose a pose classification threshold...)
The model would perform better when given similar data as its trained on.i think you should try mixing the two datasets and check the results.
To the client, you can show the three results and explain how it is a great product.
In place of mAp, show a chart of precision and recall to explain the accuracy.