title: Keep the word frequency and inverse for one type of documents
body: <p>Code example to keep the term and inverse frequency:</p>
  <pre><code>library(dplyr)
  library(janeaustenr)
  library(tidytext)
  
  book_words &lt;- austen_books() %&gt;%
      unnest_tokens(word, text) %&gt;%
      count(book, word, sort = TRUE)
  
  total_words &lt;- book_words %&gt;% 
      group_by(book) %&gt;% 
      summarize(total = sum(n))
  
  book_words &lt;- left_join(book_words, total_words)
  
  book_words &lt;- book_words %&gt;%
      bind_tf_idf(word, book, n)
  
  book_words %&gt;%
      select(-total) %&gt;%
      arrange(desc(tf_idf))
  </code></pre>
  <p>My problem is that this example uses multiple books.</p>
  <p>I have different data structure:</p>
  <pre><code>dataset1 &lt;- data.frame( anumber = c(1,2,3), text = c(&quot;Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book.&quot;,&quot;It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum&quot;, &quot;Contrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old. Richard McClintock, a Latin professor at Hampden-Sydney College in Virginia, looked up one of the more obscure Latin words, consectetur, from a Lorem Ipsum passage, and going through the cites of the word in classical literature, discovered the undoubtable source.&quot;))
  </code></pre>
  <p>In my case of dataset1 every row is one unique document. I would like to have the same result of term and inverse term frq but I don't know how to make it using my option. How can I start?</p>
  <p>Alternative option. From the calcuation of term frequency like this:</p>
  <pre><code>library(quanteda)
  myDfm &lt;- dataset1$text %&gt;%
      corpus() %&gt;%                    
      tokens(remove_punct = TRUE, remove_numbers = TRUE, remove_symbols = TRUE) %&gt;%
      tokens_ngrams(n = 1:2) %&gt;%
      dfm()
  </code></pre>
  <p>How is it possible to receive the same result as the one of tidytext, I mean of every word have the score of tf idf, using quanteda package?</p>
  <p>What I tried</p>
  <pre><code>number_of_docs &lt;- nrow(myDfm)
  term_in_docs &lt;- colSums(myDfm &gt; 0)
  idf &lt;- log2(number_of_docs / term_in_docs)
  
  # Compute TF
  
  tf &lt;- as.vector(myDfm)
  
  # Compute TF-IDF
  tf_idf &lt;- tf * idf
  names(tf_idf) &lt;- colnames(myDfm)
  sort(tf_idf, decreasing = T)[1:5]
  </code></pre>
  <p>Is the right option to receive the tf_idf using quanteda for every word frequency?</p>
  <p>to receive as output the word, term frequency, tf_idf value</p>