Key pixels, key "features" detection in CNNs I am working on a dataset but I don't know what the labels mean.
I was wondering if using CNNs there was a way to understand which pixels where most significant for the network.
A little bit in the way that running an SVM would show which pixels are the most usefull for the classification. 
Or, if there is any way to understand what my  Deep Learning algorithm is training on? 

To give a little more detail I'm using VGG on the celebA dataset getting 85% accuracy on test-set. Would you know how to get back to the label's meaning and therefore maybe crop part of the image that is useless.
Class Activation Map is what I was looking for