Scikit Learn Logistic Regression Memory Leak
Ok, this ended up being an RTFM situation, although in this case it was RTF error message.

While running this, I kept getting the following error:

DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().


I assumed that, since this had to do with the target vector, and since it was a warning only, that it would just silently change my target vector to 1-D.

However, when I explicitly converted my target vector to 1-D, my memory problems went away. Apparently having the target vector in an incorrect form caused it to convert my input vectors into dense vectors from sparse vectors.

Lesson learned: follow the recommendations when sklearn 'suggests' you do something.