How do I produce generative neural networks?
Feed-forward neural networks are discriminative models, i.e. they model $P(y|x)$.

Something like restricted Boltzmann machines look a lot like neural networks, and share some of the proprieties, and are generative models, i.e. they model $P(y,x)$ --- which, of course, you can always turn into a $P(y|x)$ using Bayes' rule, but the opposite is not true.

Libraries like Theano have support for RBMs.

EDIT: since the writing of my reply, generative adversarial networks have been invented. (Actually, they were invented in 2014, but only in 2016 were they taken to their full potential.) In these networks, you feed noise to the neural network and it generates data samples. This is achieved by co-training it against a regular discriminative neural network. Both networks are trained at the same time: the discriminative one must classify each sample as being either true or synthetically created by the other neural network. And the generative neural network must fool this one, so they get better and better, and, in the end, you have both a discriminative and a generative model!