Multi-image superresolution using CNNs
I would assume this is a lot more complicated, because the multiple images might be off by a fraction of a pixel, or taken from a marginally different angle.


I agree with your analysis that this problem presents different challenges than single-image super-resolution.

In the single image case, we know exactly where everything should be in the output image, because it is just a higher resolution version of the input.

So why not try and extend this idea to your case? Here is how you might do that:


Concatenate each low-res image and a low-res version of the output image as each training and test example
For prediction concatenate each low-res image and duplicate one of them to specify the desired orientation of the output


So if you have n 3-channel h x w input images in each training example, your input shape should be (h, w, 3*(n+1))

This should give the network enough information to learn the correct transformation.