title: Calculate TF-IDF using sklearn for n-grams in python
body: <p>I have a vocabulary list that include n-grams as follows. </p>
  
  <pre><code>myvocabulary = ['tim tam', 'jam', 'fresh milk', 'chocolates', 'biscuit pudding']
  </code></pre>
  
  <p>I want to use these words to calculate TF-IDF values.</p>
  
  <p>I also have a dictionary of corpus as follows (key = recipe number, value = recipe).</p>
  
  <pre><code>corpus = {1: "making chocolates biscuit pudding easy first get your favourite biscuit chocolates", 2: "tim tam drink new recipe that yummy and tasty more thicker than typical milkshake that uses normal chocolates", 3: "making chocolates drink different way using fresh milk egg"}
  </code></pre>
  
  <p>I am currently using the following code.</p>
  
  <pre><code>from sklearn.feature_extraction.text import TfidfVectorizer
  
  tfidf = TfidfVectorizer(vocabulary = myvocabulary, stop_words = 'english')
  tfs = tfidf.fit_transform(corpus.values())
  </code></pre>
  
  <p>Now I am printing tokens or n-grams of the recipe 1 in <code>corpus</code> along with the tF-IDF value as follows.</p>
  
  <pre><code>feature_names = tfidf.get_feature_names()
  doc = 0
  feature_index = tfs[doc,:].nonzero()[1]
  tfidf_scores = zip(feature_index, [tfs[doc, x] for x in feature_index])
  for w, s in [(feature_names[i], s) for (i, s) in tfidf_scores]:
    print(w, s)
  </code></pre>
  
  <p>The results I get is <code>chocolates 1.0</code>. However, my code does not detect n-grams (bigrams) such as <code>biscuit pudding</code> when calculating TF-IDF values. Please let me know where I make the code wrong.</p>
  
  <p>I want to get the TD-IDF matrix for <code>myvocabulary</code> terms by using the recipe documents in the <code>corpus</code>. In other words, the rows of the matrix represents <code>myvocabulary</code> and the columns of the matrix represents the recipe documents of my <code>corpus</code>. Please help me.</p>