title: monitoring stock prices on yahoo finance efficiently
body: <p>I am currently trying to get live stock prices using the beautifulsoup and requests library from the yahoo finance website. I'm currently finding that the bottleneck for speed is that the request for the webpage takes around 0.5 seconds on average. Below is my code and output</p>
  <pre><code>from bs4 import BeautifulSoup, SoupStrainer
  import requests
  import time
  
  # transform percent text into numerical (float) value
  def process_percent_change(percent_text):
      processed_text = []
      for c in percent_text:
          if c == '.' or c == '+' or c == '-' or  c.isdigit():
              processed_text.append(c)
      if processed_text[0] == '-':
          return -float(&quot;&quot;.join(processed_text[1:]))
      else:
          return float(&quot;&quot;.join(processed_text[1:]))
  
  
  # parse most current ticker data from beautiful soup object
  def get_current_data():
      # define the page to be parsed for data
      spy_link = &quot;https://finance.yahoo.com/quote/SPY?p=SPY&amp;.tsrc=fin-srch&quot; # yahoo finance link to SPY etf info
      webpage_start = time.time()
      page = requests.get(spy_link).text
      webpage_end = time.time()
  
      webpage_time = webpage_end-webpage_start
      print(&quot;webpage get time: {}&quot;.format(webpage_time))
  
      parse_start = time.time()
      strainer = SoupStrainer('div', attrs = {'data-reactid': '30'}) # only get relevant div info
      data_element = BeautifulSoup(page, 'lxml', parse_only=strainer)
  
      current_data = {}
  
      # store current data inside a dictionary
      current_data[&quot;current_price&quot;] = float(data_element.find('span', attrs={&quot;data-reactid&quot;: &quot;32&quot;}).text)
      change = data_element.find('span', attrs={&quot;data-reactid&quot;: &quot;33&quot;}).text.split()
      current_data[&quot;value_change&quot;] = float(change[0])
      current_data[&quot;percent_change&quot;] = process_percent_change(change[1])
      parse_end = time.time()
  
      parse_time = parse_end-parse_start
  
      print(&quot;parse time: {}&quot;.format(parse_time))
      # return data to be used
      return current_data
  
  if __name__ == &quot;__main__&quot;:
      current_data = get_current_data()
  </code></pre>
  <pre><code>webpage get time: 0.42667412757873535
  parse time: 0.05974388122558594
  </code></pre>
  <p>What I want to do is either speed up the request or find a more efficient way of monitoring the website for changes. I've noticed that after loading the website in my browser that the price will change without having to refresh the page. Here is a gif example for what I mean:
  <a href="https://media.giphy.com/media/nZmbqDeu7rWT2vmrks/giphy.gif" rel="nofollow noreferrer">https://media.giphy.com/media/nZmbqDeu7rWT2vmrks/giphy.gif</a></p>
  <p>Here's a link to the specific yahoo finance page: <a href="https://finance.yahoo.com/quote/SPY?p=SPY&amp;.tsrc=fin-srch" rel="nofollow noreferrer">https://finance.yahoo.com/quote/SPY?p=SPY&amp;.tsrc=fin-srch</a></p>
  <p>Is there a way to monitor the stock price such as in the gif/web browser example? If there isn't, is there a way to speed up requests (without buying a faster connection)?</p>
  <p>Any help is appreciated.</p>