Training Machine Learning in Production
There is a field of online machine learning where incremental updates can be made to a model, including a variation of stochastic gradient descent called incremental stochastic gradient descent.

Generally, several rows of data are batched together to perform an update to the parameters. Training on a single row of data at a time can lead to unstable updates.