How to visualize convolutional network focus on autoencoder? I'm trying to create some cool (and at least somewhat meaningful) visualizations of my CNN that I am using to autoencode images, as I use the encoded middle layer to do similarity comparisons.
I.e. I want a view of the input image but with some sort of &quot;heat map&quot; or something overlayed that shows what the CNN is &quot;focusing on&quot;, if that makes sense. Some sort of gradient magnitude perhaps, I saw this on a different CNN project, e.g. analogous to the examples on this site.
How could I go about doing this for my autoencoder? Network is coded in Python with Keras, and I've reproduced the code below:
img = Input(shape=(IMG_W, IMG_H, IMG_C), name='input_img_') 
    network = Convolution2D(CONV_0, 3, 3, activation='elu', border_mode='same', name='conv_0_e')(img)   
    network = MaxPooling2D((2, 2), border_mode='same')(network) 
    network = Convolution2D(CONV_1, 3, 3, activation='elu', border_mode='same', name='conv_1_e')(network)   
    network = MaxPooling2D((2, 2), border_mode='same')(network) 
    network = Convolution2D(CONV_2, 3, 3, activation='elu', border_mode='same', name='conv_2_e')(network)
    network = MaxPooling2D((2, 2), border_mode='same')(network) 
    network = Convolution2D(CONV_3, 3, 3, activation='elu', border_mode='same', name='conv_3_e')(network) 
    encoded = MaxPooling2D((2, 2), border_mode='same', name='encoded')(network) 
    
    network = Convolution2D(CONV_1, 3, 3, activation='elu', border_mode='same', name='conv_0_d')(encoded)
    network = UpSampling2D((2, 2))(network)
    network = Convolution2D(CONV_1, 3, 3, activation='elu', border_mode='same', name='conv_1_d')(network)
    network = UpSampling2D((2, 2))(network) 
    network = Convolution2D(CONV_2, 3, 3, activation='elu', border_mode='same', name='conv_2_d')(network)
    network = UpSampling2D((2, 2))(network)
    network = Convolution2D(IMG_C, 3, 3, activation='elu', border_mode='same', name='conv_decoded')(network)
    decoded = UpSampling2D((2, 2))(network) 

model_inputs = img
    model_outputs = decoded 
    model = Model(input=model_inputs, output=model_outputs)
    
    model.compile(loss='binary_crossentropy', optimizer='adadelta')

This question is a little more open-ended, I don't think there's a &quot;right answer&quot; per se, but I'm very open to creative ideas of how I could visualize what the network is doing. Hate to put it this way, but I'm looking for something that I would show a layman and they'd be like &quot;wow look at that AI go&quot; lol. But seriously that's what I'm looking for here. Maybe a collaged image of feature maps? Let me know what you think, and I'd really appreciate any sample code!
You can visualise the activation maps of the various layers, to show how a deep CNN decomposes your input into more and more abstract building blocks. There is a good discussion on Cross Validated SE here. This explains the main idea and intuition as to how CNNs work and normally gets layman motivated to listen and learn more  :-)



There is another great idea names saliency maps - [Simonyan et al.]. As well as reading that paper, I suggest watching Lecture 12 of CS231n (slides here). The general idea of 'how to do this' is, I believe, can really be applied to most deep neural networks.

Here is a sample from the linked paper:



The steps are something like this:


take a trained network architecture
choose the layer that you are interested in (be it the final layer or an intermediate layer
set all weights in layers prior to your chosen layer to zero
perform backpropagation rom your selected layer back to your input


Now you will have a set of weights that match the dimensions of your input, but the values will be those of the weights at your selected layer, simply extrapolated backwards onto your input.

You can then use these weights with an input image, perhaps simply add these value to one or all of the colour channels, and you will have an image which has certain regions highlighted. The highlighted regions signify where the corresponding neurons of your selected layer are activated - i.e. what they focus on to make their contribution towards the final prediction.



Yet another approach is to systematically occlude (cover up) parts of your images, and track how the predictions fluctuate. The intuition is that, when you cover up a certain part of an image, and the prediction becomes terrible, you know your network really looks for that part, which you just covered up. The first time this was really published (as far as I can find) was in the 2013 paper from Zeiler and Fergus.

In the image below (from that linked paper), you can see how the grey square was shifted over and image to create a final heatmap, which highlights the critical portions of the image to getting the correct image classification.