title: BS4 Scraping Hidden Content
body: <p>I have been stuck on this for awhile... I am trying to scrape the player name and projection from this site: <a href="https://www.fantasysportsco.com/Projections/Sport/MLB/Site/DraftKings/PID/793" rel="nofollow noreferrer">https://www.fantasysportsco.com/Projections/Sport/MLB/Site/DraftKings/PID/793</a></p>
  
  <p>The script is going to loop through the past by just going through all the PID's in a range, but that isnt the problem. The main problem is when I inspect the element I find the value is stored within this class:</p>
  
  <pre><code>&lt;div class="salarybox expanded"...
  </code></pre>
  
  <p>which is located in the 5th position of my projectionsView list.</p>
  
  <p>The scraper finds the projectionsView class fine but can't find anything within it.</p>
  
  <p>When I goto view the actual HTML of the site it seems this content just doesn't exsist within it..</p>
  
  <pre><code>&lt;div id="salData" class="projectionsView"&gt;
      &lt;!-- Fill in with Salary Data --&gt;
  &lt;/div&gt;
  </code></pre>
  
  <p>I'm super new to scraping and have successfully scraped everything else I need for my project just not this damn site... I think it may be because I have to sign up for the site? But either way the information is viewable without signing in so I figured I didnt need to use Selenium, and even if I did that wouldn't find it I don't think.</p>
  
  <p>Anyway here's the code I have so far that is obviously returning a blank list.</p>
  
  <pre><code>from bs4 import BeautifulSoup as soup
  from urllib.request import urlopen as uReq
  import pandas as pd 
  import os
  
  
  url = "https://www.fantasysportsco.com/Projections/Sport/MLB/Site/DraftKings/PID/793"
  
  uClient = uReq(url)
  page_read = uClient.read()
  uClient.close()
  
  page_soup = soup(page_read, "html.parser")
  salarybox = page_soup.findAll("div",{"class":"projectionsView"})
  print(salarybox[4].findAll("div",{"class":"salarybox expanded"}))
  </code></pre>
  
  <p>Any ideas would be greatly appreciated!</p>
  
  <p>The whole idea of the script is to just find the ppText of each "salarybox expanded" class on each page. I just want to know how to find these elements. Perhaps a different parser?</p>