title: using rvest and purrr::map_df to build a dataframe: dealing with multiple-element tags
body: <p>(building on my own question and its answer by @astrofunkswag <a href="https://stackoverflow.com/questions/55961475/r-using-rvest-and-purrrmap-df-to-build-a-data-frame-how-to-deal-with-incomple">here</a>)</p>
  
  <p>I am webscraping webpages with <code>rvest</code> and turning the collected data into a dataframe using <code>purrr::map_df</code>. I run into the problem that <code>map_df</code> selects only the first element of html tags with multiple elements. Ideally, I would like all elements of a tag to be captured in the resulting dataframe, and the tags with fewer elements to be recycled. </p>
  
  <p>Take the following code:</p>
  
  <pre><code>library(rvest)
  library(tidyverse)
  
  urls &lt;- list("https://en.wikipedia.org/wiki/FC_Barcelona",
               "https://en.wikipedia.org/wiki/Rome")
  h &lt;- urls %&gt;% map(read_html)
  
  out &lt;- h %&gt;% map_df(~{
    a &lt;- html_nodes(., "#firstHeading") %&gt;% html_text()
    b &lt;- html_nodes(., ".toctext") %&gt;% html_text()
  
    a &lt;- ifelse(length(a) == 0, NA, a)
    b &lt;- ifelse(length(b) == 0, NA, b)
  
    df &lt;- tibble(a, b)
  })
  out
  </code></pre>
  
  <p>which produces the following output:</p>
  
  <pre><code>&gt; out
  # A tibble: 2 x 2
    a            b        
    &lt;chr&gt;        &lt;chr&gt;    
  1 FC Barcelona History  
  2 Rome         Etymology
  &gt; 
  </code></pre>
  
  <p>This output is not desired, because it includes only the first element of the tags corresponding to <code>b</code>. In the source webpages, the elements associated to <code>b</code> are the subtitles of the webpage. The desired output looks more or less like this:</p>
  
  <pre><code>  a            b        
    &lt;chr&gt;        &lt;chr&gt;    
  1 FC Barcelona History  
  2 FC Barcelona  1899–1922: Beginnings  
  3 FC Barcelona 1923–1957: Rivera, Republic and Civil War  
  .
  .
  6 Rome         Etymology
  7 Rome         History
  8 Rome         Earliest history
  .
  .
  &gt; 
  </code></pre>