title: How to select multiple (numerical & text) columns using sklearn Pipeline & FeatureUnion for text classification?
body: <p>I have developed a text model for multilabel classification. The <a href="http://scikit-learn.org/stable/modules/generated/sklearn.multiclass.OneVsRestClassifier.html" rel="nofollow noreferrer">OneVsRestClassifier</a> LinearSVC model uses sklearns <code>Pipeline</code> and <code>FeatureUnion</code> for model preparation.</p>
  
  <p>The primary input features consist of a text column called <code>response</code> but also 5 topic probabilities (generated from a previous LDA Topic Model) called <code>t1_prob</code> - <code>t5_prob</code> to predict the 5 possible labels. There are other feature creation steps in the pipeline for the the generation of the <code>TfidfVectorizer</code>. </p>
  
  <p>I ended up calling each column with <a href="http://scikit-learn.org/stable/auto_examples/hetero_feature_union.html" rel="nofollow noreferrer">ItemSelector</a> and performing the ArrayCaster (see the code below for function definition) 5 times on these topic probability columns individually. Is there a better way to use <a href="http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.FeatureUnion.html#sklearn.pipeline.FeatureUnion" rel="nofollow noreferrer">FeatureUnion</a> to select multiple columns in a pipeline? (so I don't have to do it 5 times) </p>
  
  <p>I am wondering if it is necessary to duplicate the <code>topic1_feature</code> -<code>topic5_feature</code> code or if multiple columns can be selected in a more concise way?</p>
  
  <p>The data I am feeding in is a Pandas dataFrame:</p>
  
  <pre><code>id response label_1 label_2 label3  label_4 label_5     t1_prob t2_prob t3_prob t4_prob t5_prob
  1   Text from response...   0.0 0.0 0.0 0.0 0.0 0.0     0.0625  0.0625  0.1875  0.0625  0.1250
  2   Text to model with...   0.0 0.0 0.0 0.0 0.0 0.0     0.1333  0.1333  0.0667  0.0667  0.0667  
  3   Text to work with ...   0.0 0.0 0.0 0.0 0.0 0.0     0.1111  0.0938  0.0393  0.0198  0.2759  
  4   Free text comments ...  0.0 0.0 1.0 1.0 0.0 0.0     0.2162  0.1104  0.0341  0.0847  0.0559  
  </code></pre>
  
  <p>The x_train is <code>response</code> and the 5 topic probability columns (t1_prob, t2_prob,    t3_prob,    t4_prob,    t5_prob).</p>
  
  <p>The y_train is the 5 <code>label</code> columns which I have called <code>.values</code> on to return a numpy representation of the DataFrame. (label_1,  label_2,    label3, label_4,    label_5)</p>
  
  <p>Sample DataFrame:</p>
  
  <pre><code>import pandas as pd
  column_headers = ["id", "response", 
                    "label_1", "label_2", "label3", "label_4", "label_5",
                    "t1_prob", "t2_prob", "t3_prob", "t4_prob", "t5_prob"]
  
  input_data = [
      [1, "Text from response",0.0,0.0,1.0,0.0,0.0,0.0625,0.0625,0.1875,0.0625,0.1250],
      [2, "Text to model with",0.0,0.0,0.0,0.0,0.0,0.1333,0.1333,0.0667,0.0667,0.0667],
      [3, "Text to work with",0.0,0.0,0.0,0.0,0.0,0.1111,0.0938,0.0393,0.0198,0.2759],
      [4, "Free text comments",0.0,0.0,1.0,1.0,1.0,0.2162,0.1104,0.0341,0.0847,0.0559]
      ]
  
  df = pd.DataFrame(input_data, columns = column_headers)
  df = df.set_index('id')
  df
  </code></pre>
  
  <p>I think my implementation is a little bit round about because FeatureUnion will only handle 2-D arrays when combining them, so any other type like DataFrame have been problematic for me. However, this example works--I am just looking for ways to improve it and make it more DRY.</p>
  
  <pre><code>from sklearn.pipeline import Pipeline, FeatureUnion
  from sklearn.base import BaseEstimator, TransformerMixin
  
  class ItemSelector(BaseEstimator, TransformerMixin):
      def __init__(self, column):
          self.column = column
  
      def fit(self, X, y=None):
          return self
  
      def transform(self, X, y=None):
          return X[self.column]
  
  class ArrayCaster(BaseEstimator, TransformerMixin):
      def fit(self, x, y=None):
          return self
  
      def transform(self, data):
          return np.transpose(np.matrix(data))
  
  
  def basic_text_model(trainX, testX, trainY, testY, classLabels, plotPath):
      '''OneVsRestClassifier for multi-label prediction''' 
  pipeline = Pipeline([
      ('features', FeatureUnion([
              ('topic1_feature', Pipeline([
                  ('selector', ItemSelector(column='t1_prob')),
                  ('caster', ArrayCaster())
              ])),
              ('topic2_feature', Pipeline([
                  ('selector', ItemSelector(column='t2_prob')),
                  ('caster', ArrayCaster())
              ])),
              ('topic3_feature', Pipeline([
                  ('selector', ItemSelector(column='t3_prob')),
                  ('caster', ArrayCaster())
              ])),
              ('topic4_feature', Pipeline([
                  ('selector', ItemSelector(column='t4_prob')),
                  ('caster', ArrayCaster())
              ])),
              ('topic5_feature', Pipeline([
                  ('selector', ItemSelector(column='t5_prob')),
                  ('caster', ArrayCaster())
              ])),
             ('word_features', Pipeline([
                      ('vect', CountVectorizer(analyzer="word", stop_words='english')), 
                      ('tfidf', TfidfTransformer(use_idf = True)),
              ])),
       ])),
      ('clf', OneVsRestClassifier(svm.LinearSVC(random_state=random_state))) 
  ])
  
  # Fit the model
  pipeline.fit(trainX, trainY)
  predicted = pipeline.predict(testX)
  </code></pre>
  
  <p>My incorporation of ArrayCaster into the process arose from this <a href="https://stackoverflow.com/questions/25795511/unable-to-use-featureunion-in-scikit-learn-due-to-different-dimensions">answer</a>. </p>