Calculating the average of gradient decent
Starting from the last part, as the entire dataset is used, number of epochs(run over entire dataset) equals number of iterations. Instead, one can do the calculation in "mini batches" (of 32, for example), then the run over each 32 samples is called an iteration. 

As for the rest of the question, you can chose a batch that is equal to the entire dataset - this is called "batch gradient descent"; or update after every single sample (a batch size of 1) which is "stochastic gradient descent". Any other choice is called "mini-batch gradient descent. 

Deep Learning course on Coursera offers a relatively better explanation of these matters compared to Nielsen's book or 3B1B videos. You can watch the videos for free. In particular here is the video on  Gradient Descent.