Why is my loss increasing in gradient descent?
Based on your plots, it doesn't seem to be a problem in your case (see my comment). The reason behind that spike when you increase the learning rate is very likely due to the following.

Gradient descent can be simplified using the image below.



Your goal is to reach the bottom of the bowl (the optimum) and you use your gradients to know in which direction to go (in this simplistic case, should you go left or right). The gradient tells you in which direction to go, and you can view your learning rate as the "speed" at which you move. If your learning rate is too small, it can slow down the training. If your learning rate is too high, you might go in the right direction, but go too far and end up in a higher position in the bowl than previously. That's called diverging.

Also, good to note that it could be completely normal that your loss doesn't always decrease. This is particularly true if you use mini-batch gradient descent. In that scenario, your gradient may not always be completely accurate, and you might simply make a step in the wrong direction every once in a while.

I hope this explanation helps!