Add sum of squares of model parameters to loss function
Adding sum of squares of model weights is known to be a regularization method that prevents overfitting of the deep learning models.
Currently I have this loss function for my model which is implemented in pytorch:

lossfunc = nn.NLLLoss(ignore_index=0)


Then, I compute the loss during training as follows:

 ... 
loss = lossfunc(out_perm, dec_pp[:,1:])


I want the regularized loss to look like:

loss = lossfunc(out_perm, dec_pp[:,1:])+sum_square_wights(enc)+sum_square_wights(dec)


But don't know how to implement sum_square_wights function. Any help will be appreciated.

I believe you are referring to  L2 regularization. If that is indeed the case, L2 regularization is already added to optimizer  in torch (SGD, Adam etc) and you can control it using a non-zero value for weight_decay in optimizer's parameter. 

As for as L1 regularization is concerned, something like this should do the job:

l1_criterion = nn.L1Loss(size_average=False)
l1_reg_loss = 0
for param in model.parameters():
    l1_reg_loss += l1_criterion (param)

lambda = 0.0005
loss += lambda * l1_reg_loss 

