image reconstruction using a set of kernels
I think what you would like to do is something similar to an Auto-Encoder, in which one would append an Encoder to reconstruct back the image. The 'kernel', or more appropriately the intermediary representation is used to synthesize an image, and an appropriate reconstruction objective is minimized to recover the input. On the other hand, you could always train a second upsampling network, which reconstructs the input image, given a 1D/2D feature.