title: Webscraping in R
body: <p>I'm working on a project studying state issued municipal bonds but I am having trouble getting my data.  Using the XML package and the below code I was able to get some of it.  </p>
  
  <pre><code>&gt; nys="http://newyork.municipalbonds.com/bonds/issue/649787N87"
  &gt; nys.table=readHTMLTable(nys,asText=TRUE,which=4)
  &gt; nys.table=as.data.frame(nys.table)
  &gt; head(nys.table)
    Trade Date Trade Time Maturity Date Coupon   Price Yield Trade Amount      Trade Type
  1 2012-09-27     2:49pm      2013-Apr 5.000% 102.522 0.289     $270,000 Investor bought
  2 2012-09-27     1:17pm      2013-Apr 5.000% 102.290 0.712      $45,000    Inter-dealer
  </code></pre>
  
  <p>But that site only offers a small sample for free.  The official website, EMMA has the data for free but I'm having a terrible time scraping it.  When I try the same approach as before I end up with</p>
  
  <pre><code>nys="http://emma.msrb.org/SecurityView/SecurityDetailsTrades.aspx?cusip=649787N87"
  nys.table=readHTMLTable(nys,asText=TRUE)
  nys.table=as.data.frame(nys.table)
  head(nys.table)
  
  data frame with 0 columns and 0 rows
  </code></pre>
  
  <p>From what I understand, and I'm fairly certain about this, is that there is a standard T&amp;C page when you navigate to it via web browser.  After using htmlParse(nys), the output is identical to the page source code of the T&amp;C page and not the page where the data is actually located.  So when the code runs, it is trying to find tables on the T&amp;C page.</p>
  
  <p>I figured that this would be a fairly common problem but so far I have not been able to find any posts where someone had a similar issue.  If someone could point me in the right direction, I'd be greatly appreciative.   </p>