How to update weights in a neural network using gradient descent with mini-batches?
Let us say that the output of one neural network given it's parameters is $$f(x;w)$$
Let us define the loss function as the squared L2 loss (in this case).
$$L(X,y;w) = \frac{1}{2n}\sum_{i=0}^{n}[f(X_i;w)-y_i]^2$$
In this case the batchsize will be denoted as $n$. Essentially what this means is that we iterate over a finite subset of samples with the size of the subset being equal to your batch-size, and use the gradient normalized under this batch. We do this until we have exhausted every data-point in the dataset. Then the epoch is over. The gradient in this case:
$$\frac{\partial L(X,y;w)}{\partial w} = \frac{1}{n}\sum_{i=0}^{n}[f(X_i;w)-y_i]\frac{\partial f(X_i;w)}{\partial w}$$
Using batch gradient descent normalizes your gradient, so the updates are not as sporadic as if you have used stochastic gradient descent.