Why convolute if Max Pooling is just going to downsample the image anyway? The idea of applying filters to do something like identify edges, is a pretty cool idea.  

For example, you can take an image of a 7. With some filters, you can end up with transformed images that emphasize different characteristics of the original image. The original 7:



can be experienced by the network as:  

  

Notice how each image has extracted a different edge of the original 7.  

This is all great, but then, say the next layer in your network is a Max Pooling layer.  

My question is, generally, doesn't this seem a little bit like overkill? We just were very careful and deliberate with identifying edges using filters -- now, we no longer care about any of that, since we've blasted the hell out of the pixel values! Please correct me if I'm wrong, but we went from 25 X 25 to 2 X 2! Why not just go straight to Max Pooling then, won't we end up with basically the same thing?

As an extension the my question, I can't help but wonder what would happen if, coincidentally, each of the 4 squares all just happen to have a pixel with the same max value. Surely this isn't a rare case, right? Suddenly all your training images look the exact same.
Max pooling doesn't down-sample the image. It down-samples the features (such as edges) that you have just extracted. Which means you get more approximately where those edges or other features are. Often this is just what the network needs for generalisation - in order to classify it doesn't need to know there is a vertical edge running from 10,5 to 10,20, but that there is an approximately vertical edge about 1/3 from left edge about 2/3 height of the image. 

These rougher categories of features inherently cover more variations in the input image for very little cost, and the reduction in size of the feature map is a nice side effect too, making the network faster.

For this to work well, you still need to extract features to start with, which max pooling does not do, so the convolutional layer is necessary. You should find you can down-sample the original image (to 14x14) instead of using the first max-pooling layer, and you will still get pretty reasonable accuracy. How much pooling to do, and where to add those layers is yet another hyper-parameter problem when building a deep neural network.