GAN - am I seeing mode collapse? Common fixes not working
Since this question I have made significant progress, which I have written up as part 3 of the series:

http://makeyourownalgorithmicart.blogspot.com/2019/05/generative-adversarial-networks-part-iii.html

Although my GAN doesn't now have mode collapse - the reasons are still unclear. The post reports visually the impact of combinations of SGD->Adam, none->LayerNorm, Sigmoid->ReLU which I hope others find useful: