Can we use prior information to improve the results of a neural network? If so, how do we incorporate it into the learning process?
What you may be referring to is transfer learning.
The process of transfer learning mainly boils down to using layers of big  pre-trained models (super set) in your smaller model (subset) during training.
Additional configuration may include freezing some of the pretrained layers (no updates during training)  and unfreezing others (update weights/biases during backprop).
It is very common in recurrent networks to use pre-trained word embedding layers (word2vec, GloVe).