inserting training instances to scikit-learn dataset
FIRST: I'm guessing the reason sparse data is giving a different answer than the same data converted to dense, is that my representation of sparse was starting feature indices from one rather than zero (because oll library that I used previously required so). So my first column was all zero, when converted to dense it was not preserved and that's the reason for slightly better results when using dense representation.
SECOND: adding new rows to the sparse matrix in that scale is not efficient. Not even if you reserve a large matrix at the beginning (with padded zeros) to replace later. This can be because of the structure sparse matrix is stored in (It uses three arrays, in case of crs one for row number, one for non-zero column indices in rows and one for values themselves; check the documentation).
SOLUTION: best way I found is to use dense representations from the beginning (if that's an option of course). Collect all the instances that need to be added to the training set. Instantiate a new matrix to the size of aggregated data and then start adding instances "randomly" from both last training set and also from new instances that you want to add. To make it random I generate a sorted list of random positions that tell me when I should add data from new instances otherwise copy from the older ones.
I have a dataset of 15M+ training instances in form of svmlight dataset. I read these data using sklearn.datasets.load_svmlight_file(). The data itself is not sparse, so I don't mind converting it to any other dense representation (I will prefer that).
At some point in my program I need to add millions of new data records (instances) to my training data (in random positions). I used vstack and also tried converting to dense matrices but was either inefficient or failed to give correct results (details below). Is there any way to do this task efficiently?
I'm implementing DAgger algorithm and in the first iteration it is trying to add about 7M new training instances. I want to add these new instances in random positions. I tried vstack (given my data was in csr format I was expecting it not to be too inefficient at least). However after 24hours it's not done yet. 
I tried converting my data to numpy.matrix format just after loading them in svmlight format. A sampling showed it can help me speed things up but interestingly the results I get from training on the converted dataset and the original dataset seem not to match with each other. It appears sklearn does not work with numpy matrix in the way I thought. I couldn't find anything in the sklearn documentation.
Another approach I thought was to define a larger dataset from the beginning so that it will "reserve" enough space in memory, but when I'm using sklearn train or test features I'll index my dataset to the last "true" record. In this way, I presume, vstack will not require opening up a new large space in memory which can make the whole operation take longer. Any thoughts on this?