How to measure the reasoning capabilities of neural networks
What do you mean by "understood the concept of a car" ? A classification algorithm need not know that cars are used to transport humans or anything else it's used for. It only needs to know what combination of features we decide to label as a car. 

Or maybe you meant we could use some other metric for understanding that is not classification accuracy?

This is a kind of Catch-22 situation. 

When we want to classify a car we can reformulate the task to learning what set of a combination of features do us humans decide to call a car. What labels we want to assign a combination of features is arbitrary to a degree. It preferably has to be  consistent for inference but an NN can learn a dataset assigned completely random features to a 100% accuracy.

For instance, perhaps in an alternative universe we label the front-view of a car as 'B' and the rear view as 'A' and the notion/label of 'car' is nonexistent: even though we are ostensibly referencing the same object in physical space, we can assign it different labels. 

It follows that to measure if a model has properly interpolated what combination of features we like to call car, we ourselves must be able list out such features; and if we were able to do that, we would not not need the process of learning/teaching algorithms in the first place!

A much more interesting subject is testing the extrapolative powers of neural networks- something much more proximate to what we canonically refer to as 'reasoning'.

It's quite hard to measure this in static environments such as the classification of images, but the ability to extrapolate is more apparent in reinforcement learning settings where the model requires us to train it for times order of magnitudes above the time for needed for humans. 

The whole notion of pristine reasoning in machine learning breaks down when it comes to induction(see sufficient reason). Thus, we resort to probabilities that can be roughly translated to "this line of reasoning yielded the correct result 99.9% of the time, so it would make sense that we keep using it. 

In fact, the idea of perfect-as-possible induction has been mathematically formulated except it isn't computable :( Solomonoff Induction