Why large weights are prohibited in neural networks?
Many strategies used in machine learning are explicitly designed to reduce the test error, possibly at the expense of increased training error. Generally,
regularization is any modification we make to a
learning algorithm that is intended to reduce its generalization error but not its training error.

The L2 parameter norm penalty commonly known as Weight decay is one of the simplest and most common kinds of regularization technique which forces the weights to become smaller, by adding a parameter norm $Ω(θ) = 1/2 || w||^{2}_2$ penalty to the objective function.
For example, in linear regression, this gives us solutions that have a smaller slope, or put weight on fewer of the features. In other words, even though the model is capable of representing functions with much more complicated shape, weight decay has encouraged it to use a simpler function described by smaller coefficients.

Intuitively, in the feature space, only directions along which the parameters contribute significantly to reducing the objective function are preserved relatively intact. In directions that do not contribute to reducing the objective function, movement in this direction will not significantly increase the gradient.So, Components of the weight vector corresponding to such unimportant directions are decayed away through the use of the regularization throughout training.

Another simple explanation is when your weights are large, they are more sensitive to small noises in the input data. So, when a small noise is propagated through your network with large weights, it produces much different value in the output layer of the NN rather than a network with small weights.

Note that weight decay is not the only regularization technique. In the past few years, some other approaches have been introduced such as Dropout, Bagging, Early Stop, and Parameter Sharing which work very well in NNs.

There are other interesting findings in this very rich chapter.