title: Multithreaded download of yahoo stock history with python yfinance
body: <p>I'm trying to download the historical data for a list of tickers and export each to a csv file. I can make this work as a for loop but that is very slow when the list of stock tickers is in the 1000's. I'm trying to multithread the process but I keep getting many different errors. At times it will download just 1 file other times 2 or 3 and a few times even 6 but never beyond that. I'm guessing that has something to do with having a 6 core 12 thread processor, but I really don't know.</p>
  <pre><code>import csv
  import os
  import yfinance as yf
  import pandas as pd
  from threading import Thread
  
  ticker_list = []
  
  with open('tickers.csv', 'r') as csvfile:
      reader = csv.reader(csvfile, delimiter=',')
      name = None
      for row in reader:
          if row[0]:
              ticker_list.append(row[0])
  
  start_date = '2019-03-03'
  end_date = '2020-03-04'
  
  data = pd.DataFrame()
  
  def y_hist(i):
      ticker = ticker_list[i]
      data = yf.download(ticker, start=start_date, end=end_date, group_by=&quot;ticker&quot;)
      data.to_csv('yhist/' + ticker + '.csv', sep=',', encoding='utf-8')
  
  threads = []
  
  for i in range(os.cpu_count()):
      print('registering thread %d' % i)
      threads.append(Thread(target=y_hist,args=(i,)))
  
  for thread in threads:
      thread.start()
  
  for thread in threads:
      thread.join()
  
  print('done')
  </code></pre>
  <p>This is a sample file of the csv with the tickers just enough to test this. <a href="https://drive.google.com/open?id=1gch6PNspGQqPKlERhLjshqUcV4u8yalN" rel="nofollow noreferrer">ticker.csv</a></p>
  <p>These are the pages I've read and used code from in an attempt to make this work:</p>
  <p><a href="https://stackoverflow.com/questions/39358982/multithreading-to-scrap-yahoo-finance/39359155#39359155">multithreading-to-scrape-yahoo-finance</a></p>
  <p><a href="https://github.com/engineer-man/youtube/blob/master/036/threads.py" rel="nofollow noreferrer">Engineer Man threads</a></p>
  <p><a href="https://medium.com/velotio-perspectives/an-introduction-to-asynchronous-programming-in-python-af0189a88bbb" rel="nofollow noreferrer">an-introduction-to-asynchronous-programming-in-python</a></p>
  <p>This is a simplified version with it's output maybe it will help to clarify the issue.</p>
  <pre><code>import os
  import pandas as pd
  import yfinance as yf
  from threading import Thread
  
  ticker_list = ['IBM','MSFT','QQQ','SPY','FB','XLV','XLF','XLK','XLE','GTHX','IYR','ONE','ROG','OLED','GLD']
  
  def y_hist():
      for ticker in ticker_list:
          print(ticker)
  
  threads = []
  
  for i in range(os.cpu_count()):
      threads.append(Thread(target=y_hist))
  
  for thread in threads:
      thread.start()
  
  for thread in threads:
      thread.join()
  </code></pre>
  <p>Output:</p>
  <pre><code>IBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  GLD
  IBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  GLD
  IBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  IBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  GLD
  OLEDIBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  GLD
  IBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  IBM
  GLD
  MSFT
  ROG
  OLED
  GLD
  
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  GLD
  IBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  GLD
  IBM
  MSFT
  QQQ
  SPY
  IBM
  MSFT
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  GLD
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  GLD
  IBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  IBM
  MSFT
  QQQ
  SPY
  FB
  XLV
  XLF
  XLK
  XLE
  GTHX
  IYR
  ONE
  ROG
  OLED
  GLD
  GLD
  </code></pre>