Where is L2-regularization term applied
The regularization terms are applied to the loss functions by default.
However, their gradients do appear in the update step as the gradient of loss appears in the update step.