title: How to post a request to .aspx page with python
body: <p>I am trying to scrape the following website: <a href="https://wwwapps.ncmedboard.org/Clients/NCBOM/Public/LicenseeInformationResults.aspx" rel="nofollow noreferrer">https://wwwapps.ncmedboard.org/Clients/NCBOM/Public/LicenseeInformationResults.aspx</a></p>
  
  <p>In order to get each page to scrape, I need to first conduct a search on this .aspx page by inputting a first name and last name and initiating the search.</p>
  
  <p>Using resources on the internet, I put together the following http POST request:</p>
  
  <pre><code>url = 'https://wwwapps.ncmedboard.org/Clients/NCBOM/Public/LicenseeInformationResults.aspx'
  
  headers = {
      'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
      'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36',
      'Content-Type': 'application/x-www-form-urlencoded',
      'Accept-Encoding': 'gzip, deflate',
      'Accept-Language': 'en-GB,en;q=0.9,en-US;q=0.8,zh-TW;q=0.7,zh;q=0.6,zh-CN;q=0.5'
  }
  
  session = requests.session()
  response = session.get(url, headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36'})
  soup = BeautifulSoup(response.content, 'html.parser')
  form_data = {
      '__VIEWSTATE': soup.find('input', {'name': '__VIEWSTATE'}).get('value'),
      '__VIEWSTATEGENERATOR': soup.find('input', {'name': '__VIEWSTATEGENERATOR'}).get('value'),
      'waLastName':'Smith',
      'waFirstName':'John',
      '__EVENTTARGET':'btnNext'
  }
  
  f = session.post(url, data=form_data, headers=headers)
  soup = BeautifulSoup(f.content, 'html.parser')
  for a in soup.find_all('a', href=True):
      print("Found the URL:" + a['href'])
  </code></pre>
  
  <p>It doesn't seem like the post has any effect, since when you look at the html after the post request, it doesn't seem to show the results page. Any pointers on why this is the case?</p>
  
  <p>Thanks!</p>