GANs and grayscale imagery colorization I am currently studying colorization of grayscale satellite imagery as part of my Master's internship. After looking for various machine learning techniques, I quickly decided to go for deep learning, as the colorization process can be completely automated and gives good results. I trained a wide variety of models and architectures (CNNs, Pix2Pix, DCGANs and WGANs with UNet or residual architectures, etc.) and settled with convolutional cGANs (condition: grayscal image ; output : colored image), but end up facing the same problem every time. Indeed, when training my network, the output is quite correct but always gives me grayish roofs. 

I think it has something to do with the distribution of pixels values, as roofs are usually orange or black in my geographic area. Many articles state the fact that GANs are prone to mode collapse. In this case, there are two modes (orange and black) which probably results in a bad local equilibrium.

I tried many different techniques in order to improve training and/or avoid mode collapse :


L1 loss regularization for the generator
Label smoothing
Noise taken from a normal distribution and concatenated to every layer in the generator/discriminator
Dropout for some layers in the generator
Gradient penalty
Learning procedure inspired by DRAGANs and BEGANs


So far, gradient penalty has given me the best results, despite still having these grayish roofs. I know there are other ways of improving the training procedure (e.g. batch discrimination), but I do not have the pre-requisites in mathematics or computer science nor the time for implementing such techniques.

I was wondering if someone had some ideas for getting better results, or if someone also tried colorizing satellite imagery and ended up having such issues.

Thanks for reading this long message.

Have a nice day !
Colorfulness, specifically managing overly gray output, was touched on in the pix2pix paper, down in Section 4.2. It might be worthwhile to try out their formulation of a conditional GAN, as their results indicate that their objective function was useful in generating a more diverse set of colors.