What is the order of execution of steps in back-propagation algorithm in a neural network?
Both are feasible.
A generalization is to apply the modification of the weights after the presentation of $n$ examples. If $n=1$, this is online training. If $n=60$ ($60$, in your case, is the size of the dataset), this is batch training. In other cases, this is mini-batch training.
The main difference is not the computation complexity of the algorithm, but the theoretical speed of convergence to an &quot;optimal&quot; set of weights.
It is now generally admitted that online training is faster. Online training performs a stochastic approximation of the gradient descent method. The true gradient of the cost function over the whole training set is approximated by a gradient at each presentation of a training example.
Intuitively, with batch learning the weights are modified by an average of the gradient over the whole training set and averaging destroys information.
While with stochastic (or mini-batch learning), each example has his word to say. One after the other.
This has also been discussed here.