title: Need advice on how to speed up web scraper
body: <p>I am still pretty new to this. I am trying to pull data from web pages, but this method I have implemented seems a bit slow. I used the time module to narrow down the cause of the lag. </p>
  
  <pre><code>requests.get(url)
  </code></pre>
  
  <p>took the majority of the time (1-5 seconds)</p>
  
  <pre><code>soup = BeautifulSoup(data.content)
  </code></pre>
  
  <p>took about 0.15 seconds consistently</p>
  
  <p>Is requests always this slow? Is the problem that yahoo rate-limits requests from their servers? Right now it is taking around 2-5 seconds to pull a single stock price from yahoo.com/finance and requests is the major problem, any ideas?</p>