title: Proper HTTP POST request to mimic submitting an online form (using HTML, cURL or python)
body: <p><strong>Problem description:</strong>
  I am having trouble figuring out what is missing from my HTTP request to get to the next page after this form:
  When I do this from my code, I land on a &quot;welcome&quot; page instead of landing on the expected one from Step 2 below:</p>
  <ol>
  <li><p><strong>Step 1:</strong> Filling it out with the following and clicking &quot;Search&quot; in a browser:</p>
  </li>
  <li><p><strong>Step 2:</strong> I get to this next page:</p>
  </li>
  </ol>
  <p>I have looked up all form data being posted and did a simplified pre-filled form here, but clicking on Search in this case does not work:</p>
  <p><strong>What I have tried</strong></p>
  <ul>
  <li>I have read <a href="https://stackoverflow.com/questions/43805588/how-to-login-cookies-enabled-websites-using-curl-command-line-tool">this thread</a> which advises against simplifying the form, but I believe I am doing this correctly: all the fields I am aware of are present, I simply change their HTML visualization (which has no bearing on what is POSTed).</li>
  <li>The jspSynchronizerToken field could be checked on the server side, therefore I already tried filling it out manually with a valid value retrieved in the same browser. This alone does not solve the problem.</li>
  <li>I have tried using the following cURL command, but this does not work either:</li>
  </ul>
  <pre><code>curl -A &quot;Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0&quot; \
      -F 'locale=en_CA' \
      -F 'jspSynchronizerToken=BzDQe5n2dIhV4LHKyqV_u341627003039920' \
      -F 'FORMAT=TXT' \
      -F 'SORTED_BY=0' \
      -F 'SearchType=0' \
      -F 'Search=String With Spaces' \
      https://www.sed.com/sed?locale=en_CA \
  &gt; ~/Downloads/sed.html
  </code></pre>
  <ul>
  <li>I have also coded a more in-depth python version in which I provide the referer as well as the retrieved token from the Step 1, but with no success. (This code is available but it is simply a less readable version of the above simplified code so I don't see the point in posting, let me know if I am wrong).</li>
  <li>I have considered cookies and javascript might be the problem, yet a browser with no javascript or cookies allowed correctly submits the form and lands on the correct result.</li>
  <li>I am currently looking at the requests / responses using Charles Proxy, but I am not getting any new insights so far...</li>
  </ul>
  <p>I realize this question is a bit specific to the website I am showing, nevertheless it seems to me that there is a generic understanding of how websites process HTTP requests and how to validly mimic those which could be gained by a proper answer here. I have had other experience akin to this one in the past and am hoping to understand this properly once and for all :)</p>