Is my understanding of back-propogation correct?
Your understanding seems to be correct (although your explanation isn't completely precise), apart from &quot;adding it to your original gradient vector&quot;. You add the gradient vector to the parameters/weights vector.
(Note that back-propagation is just the algorithm that computes the gradient vector. The update of the parameters with the gradient of the loss function is the gradient descent/ascent step, even though it's true that some people, at least in the context of deep learning, refer to the combination of gradient descent and the computation of the gradients as the back-propagation algorithm, but this is just a terminology issue, which you should not dwell too much on.)

Also, how can this be done in iterations over training examples?

If I understand this question correctly, you want to know how we would update the parameters when there's more than one training example. In that case, you compute a gradient vector for each training example. Then the average of the gradient vectors (which is also a vector) is the actual gradient vector that you use to update the parameters. See this answer for more info.