title: How is the Tf-Idf value calculated with analyzer ='char'?
body: <p>I'm having a problem in understanding how we got the Tf-Idf in the following program:</p>
  
  <p>I have tried calculating the value of <code>a</code> in the document 2 (<code>'And_this_is_the_third_one.'</code>) using the concept given on the <a href="https://www.kaggle.com/divsinha/sentiment-analysis-countvectorizer-tf-idf" rel="nofollow noreferrer">site</a>, but my value of 'a' using the above concept is</p>
  
  <blockquote>
    <p>1/26*log(4/1)</p>
    
    <p>((count of occurrence of 'a' character)/(no of characters in the given
    document)*log( # Docs/ # Docs in which given character
    occurred))</p>
    
    <p>= 0.023156</p>
  </blockquote>
  
  <p>But output is returned as 0.2203 as can be seen in the output.</p>
  
  <pre><code>from sklearn.feature_extraction.text import TfidfVectorizer
  corpus = ['This_is_the_first_document.', 'This_document_is_the_second_document.', 'And_this_is_the_third_one.', 'Is_this_the_first_document?', ]
  vectorizer = TfidfVectorizer(min_df=0.0, analyzer="char")
  X = vectorizer.fit_transform(corpus)
  print(vectorizer.get_feature_names())
  print(vectorizer.vocabulary_)
  m = X.todense()
  print(m)
  </code></pre>
  
  <p>I expected the output to be 0.023156 using the concept explained above.</p>
  
  <p>The output is:</p>
  
  <pre><code>['.', '?', '_', 'a', 'c', 'd', 'e', 'f', 'h', 'i', 'm', 'n', 'o', 'r', 's', 't', 'u']
  
  
  {'t': 15, 'h': 8, 'i': 9, 's': 14, '_': 2, 'e': 6, 'f': 7, 'r': 13, 'd': 5, 'o': 12, 'c': 4, 'u': 16, 'm': 10, 'n': 11, '.': 0, 'a': 3, '?': 1}
  
  
  [[0.14540332 0.         0.47550697 0.         0.14540332 0.11887674
    0.23775349 0.17960203 0.23775349 0.35663023 0.14540332 0.11887674
    0.11887674 0.14540332 0.35663023 0.47550697 0.14540332]
  
  
   [0.10814145 0.         0.44206359 0.         0.32442434 0.26523816
    0.35365088 0.         0.17682544 0.17682544 0.21628289 0.26523816
    0.26523816 0.         0.26523816 0.35365088 0.21628289]
  
  
   [0.14061506 0.         0.57481012 0.22030066 0.         0.22992405
    0.22992405 0.         0.34488607 0.34488607 0.         0.22992405
    0.11496202 0.14061506 0.22992405 0.34488607 0.        ]
  
  
   [0.         0.2243785  0.46836004 0.         0.14321789 0.11709001
    0.23418002 0.17690259 0.23418002 0.35127003 0.14321789 0.11709001
    0.11709001 0.14321789 0.35127003 0.46836004 0.14321789]]
  </code></pre>