Weight decay in neural network
According to the book, the problem of initializing weights with too big of a standard deviation is that it is very likely to cause neurons to saturate.

But with L2 regularization, when saturation occurred only the L2 term will affect the gradient, and cause weight decay.

And when weights get small enough not to cause saturation (for example around $1 / \sqrt n$), the other term comes to affect the gradeint.

So the relative influence of L2 term decreases.

(And of course, the absolute effect of L2 term will decrease by decaying the weight.)

Why $1 / \sqrt n$?

If all of the $n$ input neurons are 1 and the standard deviations of weights are $\sigma$, the standard deviation of the input to hidden neurons will be $\sqrt{n}\sigma$.

If you want $\sqrt{n}\sigma$ to be 1 to avoid saturation, $\sigma$ should be $1 / \sqrt n$.