Parsing NBA reference with python beautiful soup
Question: So I'm trying to scrape out the miscellaneous stats table from this site
<http://www.basketball-reference.com/leagues/NBA_2016.html> using python and
beautiful soup. This is the basic code so far I just want to see if it is even
reading the table but when I do print table I just get none.

    
    
    from bs4 import BeautifulSoup
    import requests
    import pandas as pd 
    
    url = "http://www.basketball-reference.com/leagues/NBA_2016.html"
    r = requests.get(url)
    data = r.text
    soup = BeautifulSoup(data)
    
    table = soup.find('table', id='misc_stats')
    print table
    

When I inspect the html on the webpage itself, the table that I want appears
with this symbol in front `<!--` and the html text is green for the portion.
What can I do?


Answer: `<!--` is the start of a comment and `-->` is the end in html so just remove
the comments before you parse it:

    
    
    from bs4 import BeautifulSoup
    import requests
    comm = re.compile("<!--|-->")
    
    html = requests.get("http://www.basketball-reference.com/leagues/NBA_2016.html").content
    cleaned_soup = BeautifulSoup(re.sub("<!--|-->","", html))
    
    
    tableStats = cleaned_soup.find('table', {'id':'team_stats'})
    
    print(tableStats)
    

