How the combination of cross entropy loss and gradient descent penalizes and rewards
The value of the loss function depends upon the prediction (which is a function of the input data and the model parameters) and the ground truth. Gradient descent works like this: 


Initialize the model parameters in some manner.
Using the input data and current model parameters, figure out the loss value of the current network weights and biases.
Figure out how to update the weights and biases such that the loss value improves.
Update the weights and biases a certain amount based on the current learning rate.


This process is repeated until convergence (as measured in a few different ways, say improvement against a validation set).

If you want to dig in to how this all works, start with 3Blue1Brown's Neural Network series on YouTube, it's not even that long. It's a good start to get a feel for the concepts before you dig in to the math.