Which online machine learning technique to use for multi-class classification problem with multiple inputs?
I suggest using Data Stream algorithms to try on your problems since you are asking for "online learning with singular values or minibatches as the dataset is too big too load all the training data into memory."

MOA is a good choice for these algorithms. Hoeffding Trees is also a good first  choice to try.