title: How to hand-engineer features of TfidfVectorizer in Scikit-learn?
body: <p>I am trying to cluster documents by keywords. I'm using the following code to make a <code>tdidf-matrix</code>:</p>
  
  <pre><code>from sklearn.feature_extraction.text import TfidfVectorizer
  tfidf_vectorizer = TfidfVectorizer(max_df=.8, max_features=1000,
                               min_df=0.07, stop_words='english',
                               use_idf=True, tokenizer=tokenize_and_stem, 
                               ngram_range=(1,2))
  
  tfidf_matrix = tfidf_vectorizer.fit_transform(documents)
  </code></pre>
  
  <p><code>print(tfidf_matrix.shape)</code>
  returns <code>(567, 209)</code>, meaning there are 567 documents, each of which has some mixture of the 209 feature words detected by the scikit-learn TdidfVectorizer.</p>
  
  <p>Now, I used <code>terms = tfidf_vectorizer.get_feature_names()</code> to get a list of the terms. Running <code>print(len(terms))</code> gives <code>209</code> </p>
  
  <p>Many of these words are unnecessary for the task, and they add noise to the clustering. I have went through the list by hand and extracted the meaningful feature names, resulting in a new <code>terms</code> list. Now, running <code>print(len(terms))</code> gives <code>67</code></p>
  
  <p>However, running <code>tfidf_vectorizer.fit_transform(documents)</code> still gives a shape of <code>(567, 209)</code>, which means the <code>fit_transform(documents)</code> function is still using the noisy list of 209 terms rather than the hand-selected list of 67 terms. </p>
  
  <p>How can I get the <code>tfidf_vectorizer.fit_transform(documents)</code> function to run using the list of 67 hand-selected terms? I'm thinking that perhaps this will require me to add at least one function to the Scikit-Learn package on my machine, correct?</p>
  
  <p>Any help is greatly appreciated. Thanks!</p>