title: Scrape website that requires button click
body: <p>I am trying to scrape <a href="https://transparency.entsoe.eu/generation/r2/actualGenerationPerGenerationUnit/show?name=&amp;defaultValue=true&amp;viewType=TABLE&amp;areaType=BZN&amp;atch=false&amp;dateTime.dateTime=17.03.2017%2000:00%7CUTC%7CDAYTIMERANGE&amp;dateTime.endDateTime=17.03.2017%2000:00%7CUTC%7CDAYTIMERANGE&amp;area.values=CTY%7C10YBE----------2!BZN%7C10YBE----------2&amp;productionType.values=B02&amp;productionType.values=B03&amp;productionType.values=B04&amp;productionType.values=B05&amp;productionType.values=B06&amp;productionType.values=B07&amp;productionType.values=B08&amp;productionType.values=B09&amp;productionType.values=B10&amp;productionType.values=B11&amp;productionType.values=B12&amp;productionType.values=B13&amp;productionType.values=B14&amp;productionType.values=B15&amp;productionType.values=B16&amp;productionType.values=B17&amp;productionType.values=B18&amp;productionType.values=B19&amp;productionType.values=B20&amp;dateTime.timezone=UTC&amp;dateTime.timezone_input=UTC&amp;dv-datatable_length=100" rel="nofollow noreferrer">this website</a>. Unfortunately, the data that I want to scrape using rvest is hidden behind a button (the plus symbol).</p>
  
  <p>I tried to do it with the rvest package and I use the following code:</p>
  
  <pre><code>library(rvest)
  url &lt;- 'https://transparency.entsoe.eu/generation/r2/actualGenerationPerGenerationUnit/show?name=&amp;defaultValue=true&amp;viewType=TABLE&amp;areaType=BZN&amp;atch=false&amp;dateTime.dateTime=17.03.2017+00:00|UTC|DAYTIMERANGE&amp;dateTime.endDateTime=17.03.2017+00:00|UTC|DAYTIMERANGE&amp;area.values=CTY|10YBE----------2!BZN|10YBE----------2&amp;productionType.values=B02&amp;productionType.values=B03&amp;productionType.values=B04&amp;productionType.values=B05&amp;productionType.values=B06&amp;productionType.values=B07&amp;productionType.values=B08&amp;productionType.values=B09&amp;productionType.values=B10&amp;productionType.values=B11&amp;productionType.values=B12&amp;productionType.values=B13&amp;productionType.values=B14&amp;productionType.values=B15&amp;productionType.values=B16&amp;productionType.values=B17&amp;productionType.values=B18&amp;productionType.values=B19&amp;productionType.values=B20&amp;dateTime.timezone=UTC&amp;dateTime.timezone_input=UTC&amp;dv-datatable_length=100'
  
  htmlpage &lt;- html_session(url) %&gt;%
    read_html() %&gt;% 
    html_nodes(".dv-value-cell") %&gt;&gt;%
    html_table()
  </code></pre>
  
  <p>The ".dv-value-cell" is extracted from the website using the SelectorGadget (in one of the <a href="https://cran.r-project.org/web/packages/rvest/vignettes/selectorgadget.html" rel="nofollow noreferrer">vignettes</a> of rvest).</p>
  
  <p>However, before I can use this code, I still need to open the plus menu. The data inside this sub table doesn't exist before clicking the button. Therefore, the code above will return an empty value.</p>
  
  <p>I used the Chrome web development tools described in <a href="https://stackoverflow.com/questions/15603561/how-can-i-debug-a-http-post-in-chrome">this question</a> to monitor what happens when I click on the button. According to that information, I see that there is a request to the following url (shortened to only highlight the difference with the original url):</p>
  
  <pre><code>https://transparency.entsoe.eu/...&amp;dateTime.timezone_input=UTC&amp;dv-datatable-detail_22WAMERCO000010Y_22WAMERCO000008L_length=10&amp;dv-datatable_length=50&amp;detailId=22WAMERCO000010Y_22WAMERCO000008L
  </code></pre>
  
  <p>As you can see, this is the original url, but there is a small additional request. However, when I try this url in my browser, it doesn't show the desired result. I must be missing something that the website passes additionally.</p>
  
  <p>The result of this request according to Chrome is exactly the data that I'm looking for (right-click > copy > copy result). So there should be a way to just download this specific data.</p>
  
  <p>I also found <a href="https://stackoverflow.com/questions/29185501/r-language-how-to-make-a-click-on-webpage-using-rvest-or-rcurl">this question</a> about a similar problem, but unfortunately the solution is quite specific for this case and misses a general explanation.</p>
  
  <p>How can I reproduce this browser request such that I receive the same table?</p>