title: Web scraping CNBC US Treasury page
body: <p>I'm trying to scrape the CNBC US Treasury yields table <a href="https://www.cnbc.com/us-treasurys/" rel="nofollow noreferrer">https://www.cnbc.com/us-treasurys/</a> but not getting anywhere. Here's my code:</p>
  <pre><code>import requests
  from bs4 import BeautifulSoup
  url='https://www.cnbc.com/us-treasurys/'
  headers =  {'User-Agent': 'Mozilla/5.0 (Windows NT x.y; Win64; x64; rv:10.0) Gecko/20100101 Firefox/10.0 '}
  page=requests.get(url,headers=headers)
  soup=BeautifulSoup(page.content,'lxml')
  print('Page status code',page.status_code)
  #
  output=[]
  mydata=soup.find_all('td',class_=&quot;BasicTable-unchanged BasicTable-numData&quot;)
  for tag in mydata:
      output.append(tag.text.strip())
  print(output)
  </code></pre>
  <p>It doesn't produce any output even though I've carefully inspected the page using Google Chrome. Similar code for the individual quote pages such as <a href="https://www.cnbc.com/quotes/US6M" rel="nofollow noreferrer">https://www.cnbc.com/quotes/US6M</a> works fine but it's too slow to loop through all 11 maturities. I'd like to scrape the entire table at once for speed. Any thoughts how to do this?</p>