title: How to use BeautifulSoup after using Selenium to navigate to page
body: <p>I have successfully used selenium to click through a few pages on a website, and now I need BeautifulSoup to parse the data. The problem is that for some reason bs4 thinks that I am on the previous page, so none of my <code>find_all</code> calls are working. Below is a snippet of my code. The <code>calendar_search.click()</code> call is working to load the desired page. I do an exaggerated implicit wait to let the page load. Per previous SO Q&amp;As, I set the <code>html</code> and <code>soup</code> variables below. I throw in another implicit wait for good measure. When I call <code>prettify</code> I can see that I am printing html from the page I was on prior to <code>calendar_search.click()</code>. How do I load the current page? Thank you.</p>
  
  <pre><code>calendar_search.click()
  
  browser.implicitly_wait(30)
  
  html = browser.page_source
  browser.implicitly_wait(30)
  soup = bs4.BeautifulSoup(html, 'html.parser')
  
  print(soup.prettify())
  </code></pre>
  
  <p>Edit: I tried an explicit wait (see below) and the exception printed, despite the 1000 seconds! Edit2: Part of the page must have loaded because I got x[0] of a find_all list I made. I'm having trouble reproducing the results, however.</p>
  
  <pre><code>try:
      element =  WebDriverWait(browser, 
      1000).until(EC.presence_of_element_located((By.CLASS_NAME, 
      'classname')))
      print("Page is ready!")
  except TimeoutException:
      print("Loading took too much time!")
  </code></pre>