title: Unable to scrape <li> tag inside the <ol> tag using beautiful soup
body: <p>I am trying to scrape the search results i.e. li tag inside the ol tag using beautiful soup on New York Times website using the following python query-</p>
  
  <pre><code>theurl =["http://query.nytimes.com/search/sitesearch/#/Microsoft/"] 
  thepage = urllib.request.urlopen(theurl)
  soup=BeautifulSoup(thepage, "html.parser")
  for profile in soup.findAll("ol",attrs={"class":"searchResultsList flush"}):
     print(profile)
  </code></pre>
  
  <p>But it simply returns the following results,</p>
  
  <pre><code>&lt;ol class="searchResultsList flush"&gt;
  &lt;/ol&gt;
  </code></pre>
  
  <p>and doesn't return the li list tags inside the ol tag. I have tried replacing "html.parser" with "lxml" that didn't help. I am thinking it may be a security parameter that doesn't let me scrape the search results  can anyone please confirm this or has a solution to this? Thanks</p>