title: Scraping table with rvest
body: <p>I'm attempting to scrape this table of occupation codes from IPUMS: <a href="https://usa.ipums.org/usa-action/variables/OCC1950#codes_section" rel="nofollow noreferrer">https://usa.ipums.org/usa-action/variables/OCC1950#codes_section</a>.</p>
  
  <p>I have (ugly) code that works (see below) but I was trying to use <code>rvest</code> and failing. I was wondering if there is a tidier way to do it? </p>
  
  <p>Inspecting the page, I think the table has a selector <code>#dataTable &gt; table &gt; tbody</code> but I haven't seen <code>&gt;</code> in a selector before and putting that selector in <code>html_node()</code> doesn't work:</p>
  
  <pre><code>"https://usa.ipums.org/usa-action/variables/OCC1950#codes_section" %&gt;%
    read_html() %&gt;%
    html_node('#dataTable &gt; table &gt; tbody') %&gt;%
    html_table()
  
  Error in UseMethod("html_table") : 
    no applicable method for 'html_table' applied to an object of class "xml_missing"
  </code></pre>
  
  <p>Nor does using just <code>#dataTable</code> in the selector:</p>
  
  <pre><code>"https://usa.ipums.org/usa-action/variables/OCC1950#codes_section" %&gt;%
    read_html() %&gt;%
    html_node('#dataTable') %&gt;%
    html_table()
  
  Error: html_name(x) == "table" is not TRUE
  </code></pre>
  
  <p>The following does work, using <code>RCurl</code>, <code>jsonlite</code>, and <code>stringr</code> to slice through the page source. Perhaps the fact that the table seems to be in a json would explain why my <code>rvest</code> attempt fails, but I'm not sure.</p>
  
  <pre><code>library(RCurl)
  library(jsonlite)
  library(magrittr)
  library(stringr)
  
  txt &lt;- "https://usa.ipums.org/usa-action/variables/OCC1950#codes_section" %&gt;%
    getURL()
  
  ipums &lt;-
    txt %&gt;%
    str_extract(".*Farmer.*") %&gt;%
    str_extract("\\[.*false\\}\\]") %&gt;%
    fromJSON()
  </code></pre>