Is there a mathematical theory behind why MLP can classify handwritten digits?
The approximation theorem says you can approximate anything. But this is kind of meaningless in so far as you can do KNN and get an arbitrary approximation of your training data also.

Proving CNN correctly extract features is, I don't think possible. Or if it is, something involving VC theory is probably the best you can do.