How are new neural network architectures 'discovered'
Think of neural networks as math, and ask yourself how is new math discovered? New math comes from a combination of theory, experimentation and feedback from community. 

Instead of thinking about it terms of creativity vs randomness to see what new ideas stick; I prefer to think in terms of theory and experimentation. 

The current methodology for new neural net discovery is no different than any other applied field of math/engineering.