title: R getURL() returning empty string
body: <p>Sorry about the title but I couldn't think how to phrase this one.</p>
  
  <p>I am trying to scrape webpages for a study - they will be subjected to a battery of linguistic  tests eventually.</p>
  
  <p>In the meantime...</p>
  
  <pre><code>    require(RCurl)
      url1 &lt;- "http://www.coindesk.com/bitinstants-charlie-shrem-sees-bitcoin-battles-ahead"  
      url2 &lt;- "http://www.coindesk.com/terms-conditions/"
  
      html &lt;- getURL(url1)   # read in page contents
      html
      [1] ""
  
      html &lt;- getURL(url2)   # read in page contents
      html
      [1] "&lt;!DOCTYPE html&gt;\r\n&lt;!--[if lt IE 7]&gt; &lt;html class=\"no-js ie ie6 oldie\" lang=\"en\"&gt; &lt;![endif]--&gt;\r\n&lt;!--[if IE 7]&gt;    &lt;html class=\"no-js ie ie7 oldie\" lang=\"en\"&gt; &lt;![endif]--&gt;\r\n&lt;!--[if IE 8]&gt;......."
  </code></pre>
  
  <p>So given two URLs, each for different pages on the same website - the request for <code>url1</code>returns an empty string.  But <code>url2</code> works just fine.</p>
  
  <p>I've tried adding a browser agent as;</p>
  
  <pre><code>html &lt;- getURL(url1, .opts=list(useragent="Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13"))   # read in page contents
  </code></pre>
  
  <p>but that makes no difference, still an empty string.</p>
  
  <p>I'm only on day two of learning R and now I AM STUMPED!</p>
  
  <p>Can anyone suggest a reason why this is happening or a solution, </p>