Calculating the average of gradient decent I am currently studying the backpropagation process and gradient decent algorithm form the book Neural Networks and Deep Learning written by Michael Nielsen and 3Blue1Brown channel in YouTube.

My question is about calculating the gradient in gradient decent algorithm(the whole dataset as input).

I have drawn a picture that shows my understanding about how the algorithm works:


For example we have 1 million hand written digit images and through the first iteration we feed the network with this 1 million images. Then the gradient is calculated for each images, summed together and averaged before updating the weights. 

If my understanding is not wrong, this is the same thing that I saw in 3Blue1Brown channel. In this process the average of gradient is calculated with respect to the cost of each image and not the average cost of the whole data set(1 million) in one iteration, so the formula for calculating the cost has no effect here, rather its derivative is used for calculating the gradients for each image and we do not take the average of costs here.

First I want to know if this is a correct picture about how one iteration of calculating gradient decent works, second, why don't we take the derivatives of average costs with respect to the weights and biases and then take the average sum of all gradients? 

And the last question, how the number of iterations and epochs can be assigned here? can we say number of epochs is always equal to the number of iteration since the whole data is used for each iteration?
Starting from the last part, as the entire dataset is used, number of epochs(run over entire dataset) equals number of iterations. Instead, one can do the calculation in "mini batches" (of 32, for example), then the run over each 32 samples is called an iteration. 

As for the rest of the question, you can chose a batch that is equal to the entire dataset - this is called "batch gradient descent"; or update after every single sample (a batch size of 1) which is "stochastic gradient descent". Any other choice is called "mini-batch gradient descent. 

Deep Learning course on Coursera offers a relatively better explanation of these matters compared to Nielsen's book or 3B1B videos. You can watch the videos for free. In particular here is the video on  Gradient Descent.