title: How to trigger a file download using R
body: <p>I am trying to use R to trigger a file download on this site: <a href="http://www.regulomedb.org" rel="nofollow noreferrer">http://www.regulomedb.org</a>. Basically, an ID, e.g., rs33914668, is input in the form, click Submit. Then in the new page, click Download in the bottom left corner to trigger a file download. </p>
  
  <p>I have tried rvest with the help from other posts. </p>
  
  <pre><code>library(httr)
  library(rvest)
  library(tidyverse)
  
  pre_pg &lt;- read_html("http://www.regulomedb.org")
  POST(
    url = "http://www.regulomedb.org", 
    body = list(
      data = "rs33914668"
    ),
    encode =  "form"
    )
  ) -&gt; res
  pg &lt;- content(res, as="parsed")
  </code></pre>
  
  <p>By checking pg, I think I am still on the first page, not the <a href="http://www.regulomedb.org/results" rel="nofollow noreferrer">http://www.regulomedb.org/results</a>. (I don't know how to check pg list other than reading it line by line). So, I cannot reach the download button. I cannot figure out why it cannot jump to the next page. </p>
  
  <p>By learning from some other posts, I managed to download the file without using rvest. </p>
  
  <pre><code>library(httr)
  library(rvest)
  library(RCurl)
  
  session &lt;- html_session("http://www.regulomedb.org")
  form &lt;- html_form(session)[[1]]
  filledform &lt;- set_values(form, `data` = "rs33914668")
  
  session2 &lt;- submit_form(session, filledform)
  form2 &lt;- html_form(session2)[[1]]
  filledform2 &lt;- set_values(form2)
  
  thesid &lt;- filledform2[["fields"]][["sid"]]$value
  theurl &lt;- paste0('http://www.regulomedb.org/download/',thesid)
  download.file(theurl,destfile="test.bed",method="libcurl")
  </code></pre>
  
  <p>In filledform2, I found the sid. Using www.regulomedb.org/download/:sid, I can download the file. </p>
  
  <p>I am new to html or even R, and don't even know what sid is. Although I made it, I am not satisfied with the coding. So, I hope some experienced users can provide better, alternative solutions, or improve my current solution. Also, what is wrong with the POST/rvest method?  </p>