title: Fill in web form, submit and download results
body: <p>I am wanting to fill in a <a href="https://nzffdms.niwa.co.nz/search" rel="nofollow noreferrer">web form</a> and submit my query and download the resulting data. Some of the fields have the option of a drop-down menu or typing in a search query, sections can also be left blank (if all sections are left blank the entire database is downloaded), hitting the &quot;search and download&quot; button should instigate the downloading of a file.</p>
  <p>Here is what I have tried (selecting all records for species &quot;Salmo salar&quot;) based on <a href="https://stackoverflow.com/questions/27631460/how-can-i-post-a-simple-html-form-in-r">this</a> question. I used my browser (Opera) &quot;Developer Tools&quot; to inspect page elements and identify the names of all the possible fields:</p>
  <pre><code>library(httr)
  
  url &lt;- &quot;https://nzffdms.niwa.co.nz/search&quot;
  
  fd &lt;- list(
    search_catchment_no_name = &quot;&quot;,
    search_river_lake = &quot;&quot;,
    search_sampling_locality = &quot;&quot;,
    search_fishing_method = &quot;&quot;,
    search_start_year = &quot;&quot;,
    search_end_year = &quot;&quot;,
    search_species  = &quot;Salmo salar&quot;, # species of interest
    search_download_format = 1,      # select csv file format
    submit = &quot;Search and Download&quot;
  )
  
  POST(url, body = fd, encode = &quot;form&quot;)
  </code></pre>
  <p>I had hoped this would result in a csv file being downloaded (all records for species &quot;Salmo salar&quot;), but no file downloads (but outputs this (list of 10, just showing the first bit):</p>
  <pre><code>Response [https://nzffdms.niwa.co.nz/search]
  Date: 2019-10-02 23:35
  Status: 200
  Content-Type: text/html; charset=utf-8
  Size: 19.1 kB
  &lt;!DOCTYPE html&gt;  
    &lt;html&gt;  
    &lt;head&gt;  
    &lt;meta http-equiv=&quot;Content-Type&quot; content=&quot;text/html; c...
      &lt;meta name=&quot;title&quot; content=&quot;NZ Freshwater Fish Database...
  &lt;meta name=&quot;description&quot; content=&quot;NIWA NZ Freshwater Fish...
  &lt;meta name=&quot;keywords&quot; content=&quot;NIWA, NZ, Freshwater Fish&quot; /&gt;
  &lt;meta name=&quot;language&quot; content=&quot;en&quot; /&gt;
  &lt;meta name=&quot;robots&quot; content=&quot;index, follow /&gt;
  
  ...
  </code></pre>
  <h1>Edit</h1>
  <p>I think the issue is with how I am calling the <code>Search and download</code> button, when inspecting the web-page most fields look like this:</p>
  <pre><code># end year field
  &lt;input maxlength=&quot;4&quot; class=&quot;form-control&quot; type=&quot;text&quot; name=&quot;search[end_year]&quot; id=&quot;search_end_year&quot;&gt;
  </code></pre>
  <p>But the <code>search and download</code> button elements don't have a <code>name</code> or <code>id</code> option:</p>
  <pre><code>&lt;input type=&quot;submit&quot; value=&quot;Search and Download&quot; class=&quot;btn btn-primary btn-md&quot;&gt;
  </code></pre>
  <p>Also I have just noticed there is a hidden field, maybe I need to define this?</p>
  <pre><code>&lt;input type=&quot;hidden&quot; name=&quot;search[_csrf_token]&quot; value=&quot;d1530f09c1ce8110b5163bd100cb0d67&quot; id=&quot;search__csrf_token&quot;&gt;
  </code></pre>
  <p>Any advice on how I can get the file downloading would be much appreciated.</p>
  <h1>Update - warning</h1>
  <p>As of 2021-12-1 the database being queried in the above question has been significantly updated, the information in this question no longer accurately reflects the website, and the associated answer by chinsoon12 below will no longer return a result if submitted.</p>