title: Use rvest to scrape all p after h? (or other R package)
body: <p>I am new to the world of html scraping and am having difficulty pulling in paragraphs under particular headings, using rvest in R.</p>
  
  <p>I want to scrape info from multiple sites that all have a relatively similar set up. They all have the same headings but the number of paragraphs under a heading can change. I was able to scrape specific paragraphs under a heading with the following code:</p>
  
  <pre><code>unitCode &lt;- data.frame(unit = c('SLE010', 'SLE115', 'MAA103'))
  
  html &lt;- sapply(unitCode, function(x) paste("http://www.deakin.edu.au/current-students/courses/unit.php?unit=", 
                                            x,
                                            "&amp;return_to=%2Fcurrent-students%2Fcourses%2Fcourse.php%3Fcourse%3DS323%26version%3D3", 
                                            sep = ''))
  assessment &lt;- html[3] %&gt;%
                html() %&gt;%
                html_nodes(xpath='//*[@id="main"]/div/div/p[3]') %&gt;%
                html_text()
  </code></pre>
  
  <p>The 'xpath' element pulls in the first paragraph under the assessment heading. Some of the pages have multiple paragraphs under the assessment heading which I can i get if I change the 'xpath' variable to specify them specifically, e.g. p[4] or p[5]. Unfortunately I want to iterate this process over hundreds of pages, so changing the xpath each time isn't appropriate, and I don't even know how many paragraphs there will be in every page. </p>
  
  <p>I think that pulling all &lt; p > after the heading I am interested in is the best option considering the uncertainty around the set-up of the pages. </p>
  
  <p>I was wondering if there is a way to scrape all &lt; p > after &lt; h3 >Assessment&lt; h3 > using rvest or some other R scraping package? </p>