title: how to build a webscraper in R using readLines and grep?
body: <p>I am quite new to R. I want to compile a 1-million-word corpus of newspaper articles. So I am trying to write a web scraper to retrieve newspaper articles from e.g. the guardian website: <a href="http://www.guardian.co.uk/politics/2011/oct/31/nick-clegg-investment-new-jobs" rel="nofollow">http://www.guardian.co.uk/politics/2011/oct/31/nick-clegg-investment-new-jobs</a>.</p>
  
  <p>The scraper is meant to start on one page, retrieve the article's body text, remove all tags and save it to a text file. Then it should go to the next article via the links on this page, get the article and so on until the file contains about 1 million words.</p>
  
  <p>Unfortunately, I did not get very far with my scraper.</p>
  
  <p>I used readLines() to get to the website's source and would now like to get hold of the relevant line in the code.</p>
  
  <p>The relevant section in the Guardian uses this id to mark the body text of the article:</p>
  
  <pre><code>&lt;div id="article-body-blocks"&gt;         
    &lt;p&gt;
      &lt;a href="http://www.guardian.co.uk/politics/boris"
         title="More from guardian.co.uk on Boris Johnson"&gt;Boris Johnson&lt;/a&gt;,
         the...a different approach."
    &lt;/p&gt;
  &lt;/div&gt;
  </code></pre>
  
  <p>I tried to get hold of this section using various expressions with grep and lookbehind - trying to get the line after this id - but I think it does not work across multiple lines. At least I cannot get it to work.</p>
  
  <p>Could anybody help out? It would be great if somebody could provide me with some code I can continue working on!</p>
  
  <p>Thanks.</p>