title: How to fill in form data, submit form, and download response ZIP file automatically in Python
body: <p>I am trying to write a Python script that can navigate to this <a href="https://www.nseindia.com/products/content/equities/equities/archieve_eq.htm" rel="nofollow noreferrer">page</a>, fill up the form data, submit the form and download a ZIP file that is returned automatically.</p>
  
  <p>the file I need is generated by selecting <strong>Bhavcopy</strong> in the form along with providing a valid date from the past.
  [A sample input and result has been shown here<a href="https://i.stack.imgur.com/tdaaz.png" rel="nofollow noreferrer">2</a></p>
  
  <p>So far I have tried to learn and implement several approaches using using Scrapy, webbrowser, requests, Beautifulsoup, Mechanize and Selenium by copying and modifying example codes from the documentations but haven't been able to make any progress.</p>
  
  <p>So far the only code that worked without errors is :</p>
  
  <pre><code>import requests
  from bs4 import BeautifulSoup
  u = 'https://www.nseindia.com/products/content/all_daily_reports.htm'
  p = requests.get(u)
  soup = BeautifulSoup(p.text, 'html.parser')
  for link in soup.find_all('a'):
  d = requests.get(link.get('href'))
  print(d)
  </code></pre>
  
  <p>This code is no where complete and I don't how to proceed.
  Logically I know I should be:</p>
  
  <ol>
  <li>fetching the page [done]</li>
  <li>selecting form elements [not done] maybe doable in Scrapy</li>
  <li>filling in data ( the 1st parameter is constant the date can be supplied in a loop) [have no clue how to do this programatically]</li>
  <li>submitting the form [same as above]</li>
  <li>clicking on the <em>a</em> tag to download the file in the <em>href</em> attribute [a request.get(on the <em>href</em> value) should do it</li>
  </ol>
  
  <p>Any pointers to how I can achieve this would be greatly appreciated.</p>
  
  <p>The webpage allows you to download a daily report called the Bhavcopy that contains the Open, Low, High, Close data of all stocks that traded in the National Stock Exchange (India) and i wish to accumulate as much historical data as possible.</p>