title: Text Analysis Using LDA and tm in R
body: <p>Hey guys I have a little bit of trouble conduction LDA because for some reason once I get ready to conduct the analysis I get errors. I'll do my best to go through what I am doing unfortunately I will not be able to provide data because the data I am using is proprietary data.</p>
  
  <p><code>dataset &lt;- read.csv("proprietarydata.csv")</code></p>
  
  <p>First I do a little bit of cleaning data$text and post are class character</p>
  
  <pre><code>dataset$text &lt;- as.character(dataset$text) 
  post &lt;- gsub("[^[:print:]]"," ",data$Post.Content)
  post &lt;- gsub("[^[:alnum:]]", " ",post)
  </code></pre>
  
  <p>post ends up looking like this:
  `</p>
  
  <pre><code>`[1] "here is a string"
   [2] "here is another string"
   etc....`
  </code></pre>
  
  <p>then I created the following function which does more cleaning:</p>
  
  <pre><code>createdtm &lt;- function(x){
  myCorpus &lt;- Corpus(VectorSource(x))
  myCorpus &lt;- tm_map(myCorpus,PlainTextDocument)
  docs &lt;- tm_map(myCorpus,tolower)
  docs &lt;- tm_map(docs, removeWords, stopwords(kind="SMART"))
  docs &lt;- tm_map(docs, removeWords, c("the"," the","will","can","regards","need","thanks","please","http"))
  docs &lt;- tm_map(docs, stripWhitespace)
  docs &lt;- tm_map(docs, PlainTextDocument)
  return(docs)}
  
  predtm &lt;- createdtm(post)
  </code></pre>
  
  <p>This end up returning a corpus that gives me something like this for every document:</p>
  
  <pre><code>[[1]]
  &lt;&lt;PlainTextDocument (metadata: 7)&gt;&gt;
  Here text string
  
  
  [[2]]
  &lt;&lt;PlainTextDocument (metadata: 7)&gt;&gt;
  Here another string
  </code></pre>
  
  <p>Then I set myself up to get ready for LDA by creating a DocumentTermMatrix</p>
  
  <pre><code>dtm &lt;- DocumentTermMatrix(predtm)
  inspect(dtm)
  
  
  &lt;&lt;DocumentTermMatrix (documents: 14640, terms: 39972)&gt;&gt;
  Non-/sparse entries: 381476/584808604
  Sparsity           : 100%
  Maximal term length: 86
  Weighting          : term frequency (tf)
  
  Docs           truclientrre truddy trudi trudy true truebegin truecontrol
                Terms
  Docs           truecrypt truecryptas trueimage truely truethis trulibraryref
                Terms
  Docs           trumored truncate truncated truncatememory truncates
                Terms
  Docs           truncatetableinautonomoustrx truncating trunk trunkhyper
                Terms
  Docs           trunking trunkread trunks trunkswitch truss trust trustashtml
                Terms
  Docs           trusted trustedbat trustedclient trustedclients
                Terms
  Docs           trustedclientsjks trustedclientspwd trustedpublisher
                Terms
  Docs           trustedreviews trustedsignon trusting trustiv trustlearn
                Terms
  Docs           trustmanager trustpoint trusts truststorefile truststorepass
                Terms
  Docs           trusty truth truthfully truths tryd tryed tryig tryin tryng
  </code></pre>
  
  <p>This looks really odd to me but this is how I have always done this. So I end up moving forward with this and do the following </p>
  
  <pre><code>run.lda &lt;- LDA(dtm,4)
  </code></pre>
  
  <p>This returns my first error</p>
  
  <pre><code>  Error in LDA(dtm, 4) : 
    Each row of the input matrix needs to contain at least one non-zero entry
  </code></pre>
  
  <p>After researching this error I check out this post <a href="https://stackoverflow.com/questions/13944252/remove-empty-documents-from-documenttermmatrix-in-r-topicmodels">Remove empty documents from DocumentTermMatrix in R topicmodels?</a> I assume I have everything under control and get excited so I follow the steps in the link but then</p>
  
  <p>This runs</p>
  
  <pre><code>rowTotals &lt;- apply(dtm , 1, sum)
  </code></pre>
  
  <p>This doesnt</p>
  
  <pre><code>dtm.new   &lt;- dtm[rowTotals&gt; 0]
  </code></pre>
  
  <p>it returns:</p>
  
  <pre><code>  Error in `[.simple_triplet_matrix`(dtm, rowTotals &gt; 0) : 
    Logical vector subscripting disabled for this object.
  </code></pre>
  
  <p>I know I might get heat because some of you might say this isn't reproducible example. Please feel free to ask anything about this problem. It's the best I can do.  </p>