How important is the balance between the components of a loss value?
The cost function controls the algorithm completely - the new regularization from weight decay is likely responsible for the jump in loss. If you only added a small amount of regularization and the loss ballooned, you were likely very overfit.

The solution is cross validation. The only way to know the optimal amount if regularization is to see what does the best on held-out data.