Model the predictive relationship between images
To answer my own question. A naïve approach is to train a convolutional neural net with an output layer of 4096 neurons $x_1,x_2,\ldots, x_{4096}$. Assume values in the response image are $y_1,y_2,\ldots,y_{4096}$. Let the loss function be $L=\sum_{i=1}^{4096}(x_i-y_i)^2$.