title: ML text-classification : combining dense and sparse features
body: <p>I have a dataset which has three (or could be anything more than two) columns A, B and C (C being unstructured text (i.e., a document)). The classification of the record is based not only on the text document but also on the combination of columns A, B and C. That is to say, the same document text, together with a different combination of A and B has different class in the training data.</p>
  
  <p>Text classification of C by itself is relatively easy but it is with the combination of three that I am experiencing challenge. For example, if my training data is train with the columns A, B &amp; C with C being document, I could work with C by itself as:</p>
  
  <pre><code>vectorizer = TfidfVectorizer()
  vtrain = vectorizer.fit_transform(train["C"])
  </code></pre>
  
  <p>and this would give me sparse feature matrix for feature names and vocabulary from vectorizer. But how do I use values in A and B (which are always non-zero) in combination with C to use further for classification (NB or SVM or the like)?</p>
  
  <p>I am thinking of using TruncatedSVD to make it dense to an accepted n_component, something like:</p>
  
  <p>svd = TruncatedSVD(n_components=100, n_iter=5, random_state=42) but the issue is how do I combine output from svd which is a dense matrix with that from train with columns A and B to train my model?</p>