title: How to scrape page with POST method with Python?
body: <p>I want to join together some pages that report Starlink constellation passages. As they are now, I need to access each single page manually, and cannot filter out according to time and visibility.</p>
  <p>The base page is <a href="https://heavens-above.com/StarlinkLaunchPasses.aspx?lat=50&amp;lng=12&amp;loc=Somewhere" rel="nofollow noreferrer">https://heavens-above.com/StarlinkLaunchPasses.aspx?lat=50&amp;lng=12&amp;loc=Somewhere</a>.</p>
  <p><a href="https://stackoverflow.com/questions/57239651/scrape-peekyou-com-having-post-method">Scrape peekyou.com ( having POST METHOD)</a> gives me some hints, but not enough to go on my feet.</p>
  <p>This is the GET code to scrape the first page (the last Starlink launch):</p>
  <pre class="lang-py prettyprint-override"><code>import pandas as pd
  import requests
  from bs4 import BeautifulSoup
  
  res = requests.get(r&quot;https://heavens-above.com/StarlinkLaunchPasses.aspx?lat=45.61&amp;lng=15.312&amp;loc=Somewhere&amp;alt=0&amp;tz=CET&quot;)
  soup = BeautifulSoup(res.content, 'lxml')
  table = str(soup.find_all(&quot;table&quot;, {&quot;class&quot;: &quot;standardTable&quot;}))
  
  df = pd.read_html(table)[0]
  
  cols = &quot;date satellite mag s_time s_altitude s_azimuth h_time h_altitude h_azimuth e_time e_altitude e_azimuth&quot;.split()
  df.columns = cols
  
  print(df)
  </code></pre>
  <p>The other pages are requested with a POST method, clicking on the dropdown list. Here stops my (shallow) knowledge about web scraping.</p>
  <p>I see that the returned <code>res.text</code> contains the form data that I could use for the next request, but I have no idea of how to extract them:</p>
  <pre><code>&lt;form name=&quot;aspnetForm&quot; method=&quot;post&quot; action=&quot;/StarlinkLaunchPasses.aspx?lat=48.55&amp;amp;lng=11.53&amp;amp;loc=Somewhere&amp;amp;alt=0&amp;amp;tz=CET&quot; id=&quot;aspnetForm&quot;&gt;
  &lt;input type=&quot;hidden&quot; name=&quot;__VIEWSTATE&quot; id=&quot;__VIEWSTATE&quot; value=&quot;DprSo0lEG4wbQojWQ3ub7mILDflL+omP+KQ
  .../&gt;
  ...
  &lt;input type=&quot;hidden&quot; name=&quot;__VIEWSTATEGENERATOR&quot; id=&quot;__VIEWSTATEGENERATOR&quot; value=&quot;9E5B71D1&quot; /&gt;
  &lt;input type=&quot;hidden&quot; name=&quot;utcOffset&quot; id=&quot;utcOffset&quot; value=&quot;7200000&quot; /&gt;
  ...
  &lt;input type=&quot;hidden&quot; name=&quot;ctl00$cph1$hidStartUtc&quot; id=&quot;ctl00_cph1_hidStartUtc&quot; value=&quot;637211090517289358&quot; /&gt;
  ...
  # and here is the dropdown list:
  &lt;select name=&quot;ctl00$cph1$ddlLaunches&quot; id=&quot;ctl00_cph1_ddlLaunches&quot;&gt;
      &lt;option selected=&quot;selected&quot; value=&quot;2020019&quot;&gt;Starlink 5, 18 March 2020 12:16&lt;/option&gt;
      &lt;option value=&quot;2020012&quot;&gt;Starlink 4, 17 February 2020 15:06&lt;/option&gt;
      &lt;option value=&quot;2020006&quot;&gt;Starlink 3, 29 January 2020 14:07&lt;/option&gt;
      &lt;option value=&quot;2020001&quot;&gt;Starlink 2, 07 January 2020 02:19&lt;/option&gt;
      &lt;option value=&quot;2019074&quot;&gt;Starlink 1, 11 November 2019 14:56&lt;/option&gt;
      &lt;option value=&quot;2019029&quot;&gt;Starlink 0, 24 May 2019 02:30&lt;/option&gt;
  &lt;/select&gt;
  </code></pre>
  <p>Could you please help me with a pointer to a possible solution?</p>
  <p>Thank you in advance</p>