Difference between Ridge and Linear Regression From what I have understood, the Ridge Regression is just having the loss function for an optimization problem with the addition of the regularization term (L2 Norm in the case of Ridge). However I am not sure if the loss function can be described by a non-linear function or it needs to be linear. In this case, if the loss functions needs to be linear, then from what I understand the Ridge regression, is simply performing Linear regression with the addition of the L2-Norm for regularization. Please correct  me if I am wrong.
Yes, ridge regression is ordinary least squares regression with an L2 penalty term on the weights in the loss function. The loss function is not really linear in any of its terms, right? it's the squared residuals plus squares of weights. But there is no reason the loss function needs to be linear. It's helpful if it's convex, which it is here, but even that is not required to try to minimize it.