is neural networks an online algorithm by nature?
There are three training modes for neural networks


stochastic gradient descent: Adjust the weights after every single training example
batch training: Adjust the weights after going through all data (an epoch)
mini-batch training: Adjust the weights after going through a mini-batch. This is usually 128 training examples.


Most of the time, mini-batch training seems to be used.

So the answer is:

No, the neural network learning algorithm is not online algorithm.