Connection between Regularization and Gradient Descent
The fitting procedure is the one that actually finds the coefficients of the model. The regularization term is used to indirectly find the coefficients by penalizing big coefficients during the fitting procedure. A simple (albeit somewhat biased/naive) example might help illustrate this difference between regularization and gradient descent:

X, y &lt;- read input data
for different values of lambda L
    for each fold of cross-validation using X,y,L
        theta &lt;- minimize (RSS + regularization using L) via MLE/GD
        score &lt;- calculate performance of model using theta on the validation set 
    if average score across folds for L is better than the current best average score 
        L_best &lt;- L


As you can see, the fitting procedure (MLE or GD in our case) finds the best coefficients given the specific value of lambda.

As a side note, I would look at this answer here about tuning the regularization parameter, because it tends a little bit murky in terms of bias.