title: Beautifulsoup web-scraping website with drop down menu
body: <p>I am trying to scrape a site that has a drop down menu where the user can select the year for the data to display.  However, I seem to be stuck in my implementation of this.</p>
  
  <p>Here is the website URL: <a href="https://www.pgatour.com/tournaments/masters-tournament/past-results.html" rel="nofollow noreferrer">https://www.pgatour.com/tournaments/masters-tournament/past-results.html</a></p>
  
  <p>This is for a personal project to scrape golf data for each major tournament for each year.  I know how to pull the desired stats once the year has been selected.</p>
  
  <p>Here is an example of the websites html for the drop down menu</p>
  
  <pre class="lang-html prettyprint-override"><code>&lt;select name="year" id="pastResultsYearSelector" class="hasCustomSelect"
  style="-webkit-appearance: menulist-button; width: 180px; position: absolute;
  opacity: 0; height: 42px; font-size: 18px;"&gt;
              &lt;option value="2019" selected="selected"&gt;2019&lt;/option&gt;
              &lt;option value="2018"&gt;2018&lt;/option&gt;
              &lt;option value="2017"&gt;2017&lt;/option&gt;
              &lt;option value="2016"&gt;2016&lt;/option&gt;
  </code></pre>
  
  <p>Here is what I've tried so far:</p>
  
  <pre class="lang-py prettyprint-override"><code>headers = {
      'user-agent': 
      'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36'
      }
  
  data = {
      'name':'2019', 'id':'pastResultYearSelector', 'class':'hasCustomSelect',
      'style':'-webkit-appearance: menulist-button; width: 180px; position: absolute; opacity: 0; height: 42px; font-size: 18px;'
      }
  
  url = "https://www.pgatour.com/tournaments/masters-tournament/past-results.html"
  
  r = requests.post(url, data=data, headers=headers, timeout=20)
  
  soup = BeautifulSoup(r.text, 'html.parser')
  </code></pre>
  
  <p>However my request seems to be invalid, as I get a response saying the requested page was not found.</p>