How is creativity generated in a currently rule based neural network?
Your question is a bit broad. Some examples of creativity and how it's achieved (in all of them there is an ordinary convolutional network trained via supervised learning, but applied differently):


Google Deep Dream uses backpropagation to the image to tweak the image towards maximum activation in the convolutional layer. As a result, it finds lots of "things" in the image and enhances them:


Generative Adversarial Networks use two competing networks (generator and discriminator) to generate samples from the training manifold:


Neuro style method captures the style of one image via Gram matrix of convolutional layer activations and trains (another) neural network with a special loss function that minimizes the joint distance to the incoming image (content) and to the Gram matrix (style). The result: