Negative examples for a Yes/No image classification neural network
Have you tried creating a dataset with 2500 images of cars and ~2500 images of random stuff and trained it first?
if so you should try that and see how it works, if it doesnt work you might have to add more images in the "distractor" class.
Provide more information about what you find after you do this experiment and may be we can help further?