Why is my loss blowing up after adding regularization
When you use regularization, your loss will be larger because you add the regularization term. So, it is normal if your best loss without regularization is lower than your best loss with regularization.
However, adding regularization should not affect convergence in the long-term. Meaning that even though your overall loss might be larger, it should decrease after some epochs.
I see that it increases for the first few epochs. I would suggest running your model for longer and see how the loss behaves. If an epoch takes long, use a small chunk of your dataset to investigate the behaviour of the loss. It may be possible that for the first few epochs, the loss doesn't start converging quite yet.