How to find out what portions of an image is helping CNN to classify it I am working on an image classification problem using Transfer learning. Right now, I am getting an accuracy of 75% on train data and 67% in test data. Now I want to understand what portions/parts of the image are being utilized for classification graphically.
Are there any packages that can return a copy of one of the input images with markings on the most useful pixels or parts of the image?
Yes, the typical approach is to obtain the saliency map of the input, which are &quot;heatmaps&quot; of the contribution of each pixel to the final classification.
In this free online book about Explainable ML, you can find the most relevant approaches to obtain saliency maps, like vanilla gradients, together with other pixel attribution techniques.
Here you can find an end-to-end tutorial on how to implement saliency maps with Keras.