Issue with Custom object detection using tensorflow when Training on a single type of object I am training a pre built tensorflow based model for custom object detection. I want to detect only 1 type of object. I have taken lot of images from different angles and in different light conditions. I am training on K80 Nvidia GPU. Everything is working and when I train I can see the loss function falling to 0.3. But the loss values drops very quickly to under 1 when I start training. I am using SSD mobile Net as the base configuration for the model. When I try to test the model, it just draws a big square on the input image, rather than detecting the desired object in the image. Basically, it fails to detect the object.

I tried to train the model with a different set of images of mac n chesse which had lot of variations. Then the model worked fine and detected images of mac n chesse in the input image. But when I have pictures of single object then the model fails to detect. Please help me understand what I am doing wrong here
How many images are you using to train?  You might need more... let's say  at least 1000 for a single category - depending on how big your MobileNet is (there are different variants).

Are you using a pretrained MobileNet, or are you training it from scratch. If Mac'n Cheese was not a category in the original data used for a pre-trained model, then you might need to train from scratch, or only keep the first few layers of the model frozen, as they learn very general features from images that will liekly be useful for your target class (and many other). If you are unsure, search for transfer learning, to see what is possible.

The nominal value of the loss is quite hard to interpret, so values of 0.3 or 1.0 don't mean a lot without context. You can use them for comparison - so if you change something and loss values then start dropping to 0.005, you know that you are likely onto a good idea.

You can check out this tutorial notebook on MobileNet, which might give you more ideas about how to play with the network