title: Pause URL request Downloads
body: <pre><code>import urllib.request
  import re
  import csv
  import pandas as pd
  from bs4 import BeautifulSoup
  
  columns = []
  data = []
  f = open('companylist.csv')
  csv_f = csv.reader(f)
  
  for row in csv_f:
      stocklist = row
      print(stocklist)
  
      for s in stocklist:
          print('http://finance.yahoo.com/q?s='+s)
          optionsUrl = urllib.request.urlopen('http://finance.yahoo.com/q?s='+s).read()
          soup = BeautifulSoup(optionsUrl, "html.parser")
          stocksymbol = ['Symbol:', s]
          optionsTable = [stocksymbol]+[
          [x.text for x in y.parent.contents]
          for y in soup.findAll('td', attrs={'class': 'yfnc_tabledata1','rtq_table': ''})
          ]
          if not columns:
              columns = [o[0] for o in optionsTable] #list(my_df.loc[0])
          data.append(o[1] for o in optionsTable)
  
  
  
  
  # create DataFrame from data
  df = pd.DataFrame(data, columns=columns)
  df.to_csv('test.csv', index=False)
  </code></pre>
  
  <p>The scripts works fine when I have about 200 to 300 stocks, but my company list has around 6000 symbols.</p>
  
  <ol>
  <li>Is there a way I can download chunks of data, say like 200 stocks at a time, pause for while, and then resume the download again?</li>
  <li>The export is one stock at a time; how do I write 200 at a time, and append the next batch to the initial batch (for the CSV)?</li>
  </ol>