Regions /patterns in images that creates classification
I think visualizing convolutional neural networks is what you want.

CS231n is a good place to start, here is a step-by-step guide about how to visualize the activations, first-layer weights, and shows how to retrieve images which maximally activate a neuron.

Also there's a demo which visualized multiple convnets (including ResNet50, Inception v3, etc). You may learn a lot from this demo (at least I do).

If you're interesting in details, this well-known paper may help you.