Which model is better for incremental learning?
I will say, it's an either Or situation
You can pick one of &quot;Incremental/Online&quot; training Or &quot;addition of new class&quot;.
 You may do a fine-tuning approach with a Neural network by adjusting the o/p layer and training the last few layers. But this approach expects the new data to be quite similar to the training set.
KNN - Can do the online stuff but it doesn't do training. It simply calculates all the distances at the time of prediction. So, no reduction in computing. But you might have to compromise on accuracy if it is not the best
Scikit-Learn SGD Classifier can help with online training but can't support new classes

For classification, a somewhat important thing to note is that although a stateless feature extraction routine may be able to cope with new/unseen attributes, the incremental learner itself may be unable to cope with new/unseen targets classes.
SGDClassifier

Other models e.g. SVM/DT doesn't support incremental learning naturally. Though there are suggested approaches on the internet. But may not be simple. See these references.
journalofbigdata$\hspace{1cm}$
A Good SE read
I am not sure how will you figure it out if it is a new Class unless you have a separate arrangement because the model will predict it an existing Class anyway. Assuming you have a setup for that. We may try below approach -
Till the time, the prediction is within &quot;known class&quot; - do an Online training.
When the data is for a &quot;New Class&quot;, do a full-data training.
 With this approach, you can reduce the frequency of full-data training.
A simple Neural network can be a good candidate for both purposes. You can also get class probability with it.
Please see these useful links too
Creme
Keras + online learning