Convolutional Neural networks
Why don't we convolve our images against the last convolution layer and see how many of these complex feature filters get activated?


The answer is that all the layers are fully dependent on the exact features of the previous layer. The last layer is simply not capable of taking a raw image as input and outputting meaningful values. Most likely it is not even going to be able to fit to the shape of the output, it will require dozens of channels, where a colour image only has 3. If you somehow forced it to fit, then the colour image data is meaningless to it. It might still be made to output something, but it would essentially be gibberish, and unrelated to the desired outcome.

Each layer is a function of the previous layer. Ignoring the details of convolution, a neural network is essentially a composition of multiple functions (let's call them $f, g, h, i, j$ for example) so that:

$$y = j(i(h(g(f(x))))$$

You are essentially asking here, can you just do $y = j(x)$ instead of running all those functions in sequence. And the answer is no.