title: Removing non-English text from Corpus in R using tm()
body: <p>I am using <code>tm()</code> and <code>wordcloud()</code> for some basic data-mining in R, but am running into difficulties because there are non-English characters in my dataset (even though I've tried to filter out other languages based on background variables.</p>
  
  <p>Let's say that some of the lines in my TXT file (saved as UTF-8 in TextWrangler) look like this:</p>
  
  <pre><code>Special
  satisfação
  Happy
  Sad
  Potential für
  </code></pre>
  
  <p>I then read my txt file into R:</p>
  
  <pre><code>words &lt;- Corpus(DirSource("~/temp", encoding = "UTF-8"),readerControl = list(language = "lat"))
  </code></pre>
  
  <p>This yields the warning message:</p>
  
  <pre><code>Warning message:
  In readLines(y, encoding = x$Encoding) :
    incomplete final line found on '/temp/file.txt'
  </code></pre>
  
  <p>But since it's a warning, not an error, I continue to push forward.</p>
  
  <pre><code>words &lt;- tm_map(words, stripWhitespace)
  words &lt;- tm_map(words, tolower)
  </code></pre>
  
  <p>This then yields the error: </p>
  
  <pre><code>Error in FUN(X[[1L]], ...) : invalid input 'satisfa��o' in 'utf8towcs'
  </code></pre>
  
  <p>I'm open to finding ways to filter out the non-English characters either in TextWrangler or R; whatever is the most expedient. Thanks for your help!</p>