Neural Network with different input shapes
In my experience you cannot train any network with different sample size on the same batch. 
Fully convolutional network is similar to a fully connected network with fully connected layers at the end. As such any input image in the batch must have the same dims (w,h,d). 
The difference is that a fully connected layers output a single output vector for every sample in the input batch while the fully convolutional net outputs a map of probability for each class.
It has a deeper meaning than just the image size, when trying to fit any data the size of it must be absolute and cannot be changed while training. I guess you can do it in different batches as I stated but I've never tried it.
An encoder\decoder could help "rebuild" the image in a specific size.
This tip, again, is from my experience in object detection, I could be wrong :)
I'm currently designing the architecture of a neural network for the colorization of grayscale images. Later on it should be able to colorize images with different sizes and different aspect ratios. I read that this would not be possible with a common CNN. I also read that the only options are downscaling the images to one specific size or to use a big fixed size (like 3000x3000px) and to fill the remaining space black. Both of these options don't seem to be that elegant. The first one is the opposite of what I want and the second would make the neural network slower.
Then I read about Fully Convolutional Networks and that this problem would not exist there. This would be great if it really works. I would like to know why this special network can deal with different input shapes. And maybe you could show me some tensorflow code of such a network.
By the way, I thought about an Autoencoder combined with a GAN for the architecture.