Training a model sample by sample I'm training a Scikit model but it seems that in all examples, they call the fit method on the entire training set. What I want to do however is call it per sample (i.e. looping through all samples). This has multiple reasons but most importantly are


MemoryError with my huge trainingset
Training with new data instead of recompiling entire model


Yet when I loop and call the fit model self.gnb = self.gnb.fit(sample.data,labels) and then debug, the gnb model only has one class (namely the last one). So how should I approach this?
In the general case this is impossible because standard learning algorithms require all the instances at once in order to compute the model. They usually work by making calculations based on the set of instances, then only the results of these calculations are stored as a model (for instance a linear regression classifier stores only the final weights which are needed to make a prediction). 

It is however possible to train a model incrementally:


Very often this is done by simply adding the new instances to the original training set, then re-train a new model on the full set of instances (new and old ones together).
There are also actual incremental learning algorithms, including adaptations of some common learning methods. However this is not very common, I'm not aware of these being implemented in the most standard ML libraries that I know of.