title: special characters in R
body: <p>I'm working on being able to read transcripts of dialogue into R.  However I run into a bump with special characters like curly quotes en and em dashes etc.  Typically I replace these special characters in a microsoft product first with <code>replace</code>.  Typically I replace special characters with plain text but on some occasions desire to replace them with other characters (ie I replace “ ” with { }).  This is tedious and not always thorough. If I could read the transcripts into R as is and then use Encoding to switch their encoding to a recognizable unicode format, I could <code>gsub</code> them out and replace them with plain text versions.  However the file is read in in some way I don't understand.
  Here's an xlsx of what my data may look like:</p>
  
  <p><a href="http://dl.dropbox.com/u/61803503/test.xlsx" rel="nofollow">http://dl.dropbox.com/u/61803503/test.xlsx</a></p>
  
  <p>This is what is in the .xlsx file</p>
  
  <pre><code>text                              num
  “ ” curly quotes                    1
  en dash (–) and the em dash (—)     2
  ‘ ’ curly apostrophe-ugg            3
  …  ellipsis are uck in R            4
  </code></pre>
  
  <p>This can be read into R with:</p>
  
  <pre><code>URL &lt;- "http://dl.dropbox.com/u/61803503/test.xlsx"
  library(gdata) 
  z &lt;- read.xls(URL, stringsAsFactors = FALSE) 
  </code></pre>
  
  <p>The result is:</p>
  
  <pre><code>                                 text num
  1                â€œ â€ curly quotes    1
  2 en dash (â€“) and the em dash (â€”)   2
  3        â€˜ â€™ curly apostrophe-ugg   3
  4          â€¦  ellipsis are uck in R   4
  </code></pre>
  
  <p>So I tried to use <code>Encoding</code> to convert to Unicode:</p>
  
  <pre><code>iconv(z[, 1], "latin1", "UTF-8")
  </code></pre>
  
  <p>This gives:</p>
  
  <pre><code>[1] "â\u0080\u009c â\u0080\u009d curly quotes"                "en dash (â\u0080\u0093) and the em dash (â\u0080\u0094)"
  [3] "â\u0080\u0098 â\u0080\u0099 curly apostrophe-ugg"        "â\u0080¦  ellipsis are uck in R"     
  </code></pre>
  
  <p>Which makes gsubing less useful.</p>
  
  <p>What can I do to convert these special characters to distinguishable unicode so I can gsub them out appropriately?  To be more explicit I was hoping to have <code>z[1, 1]</code> read:</p>
  
  <pre><code>\u201C 2\u01D curly quotes
  </code></pre>
  
  <p>To make it even more clear my desired outcome I will webscrape the tables from a page like wikipedia's: <a href="http://en.wikipedia.org/wiki/Quotation_mark_glyphs" rel="nofollow">http://en.wikipedia.org/wiki/Quotation_mark_glyphs</a> and use the unicode reference chart to replace characters appropriately.  So I need the characters to be in unicode or some standard format that I can systematically go through and replace the characters.  Maybe it already is and I'm missing it.</p>
  
  <p>PS I don't save the files as .csv or plain text because the special characters are replaced with <code>?</code> hence the use of <code>read.xls</code>  I'm not attached to any particular method of reading in the file (ie <code>read.xls</code>) if you've got a better alternative.</p>