How to investigate what has been learnt in CNNs?
There is a famous paper about tracking what a CNN network has learnt. You can visualise to see which parts are more engaged in the classification using DeConvNet. In this paper, it was officially observed that first layers attempt to find simple lines and edges while deeper layers try to put the previous things together to make abstract concepts, like mouth, eye and such meaningful things. As an example take a look at the following image: 



I guess there are implementations of this paper that you can replace your pre-trained model with the one already exists and see what exactly is learnt by your network.


  Isn't this image a different vector?


They are different vectors. ML and DL models are for generalisation which means they should be good at test time.