title: How to Check if Click in Selenium Webdriver Changed Page in Python
body: <p>I've been reading that click() using Selenium's webdrivers is asynchronous, so I've been trying to have the webdriver wait for the click to go through before doing anything else. I am using PhantomJS as my browser.</p>
  
  <p>I have the WebDriverWait objects to wait for an element on the page to change (that's how I'm telling if the page has loaded/changed once I've clicked on something). My issue is that I keep getting TimeoutExceptions from the WebDriverWait.</p>
  
  <p>Is there something that I could do to wait for the page to load after I click something? I don't want to have to use <code>time.sleep(1)</code> because there seems to be a variable loading time, and I don't want it to sleep for too long. This is why I want to explicitly wait for the page to load.</p>
  
  <p>Here is my code for the webdriver and corresponding waiting:</p>
  
  <pre><code>import time
  from bs4 import BeautifulSoup
  from selenium import webdriver
  import selenium.webdriver.support.ui as ui
  import selenium.common.exceptions as exceptions
  
  class Webdriver():
  
      def __init__(self, wait_time=10):
          self.driver = webdriver.PhantomJS()
          self.driver.set_window_size(1200,800)
          self.wait = wait_time
  
      def click(self, element_xpath, wait_xpath, sleep_time=0):
          wait = ui.WebDriverWait(self.driver, self.wait)
          old_element = self.driver.find_element_by_xpath(wait_xpath)
          old_text = old_element.text
          self.driver.find_element_by_xpath(element_xpath).click()
          wait.until(lambda driver: element_changed(driver, wait_xpath, old_text,20))
          time.sleep(sleep_time)
  
  def element_changed(driver, element_xpath, old_element_text, timeout_seconds=10):
      pause_interval = 1
      t0 = time.time()
      while time.time() - t0 &lt; timeout_seconds:
          try:
              element = driver.find_element_by_xpath(element_xpath)
              if element.text != old_element_text:
                  return True
          except exceptions.StaleElementReferenceException:
              return True
          except exceptions.NoSuchElementException:
              pass
          time.sleep(pause_interval)
      return False
  </code></pre>
  
  <p>Here is the code that runs:</p>
  
  <pre><code>driver = Webdriver()
  url = 'http://www.atmel.com/products/microcontrollers/avr/default.aspx?tab=parameters'
  wait_xpath = '//*[@id="device-columns"]/tbody/tr[2]/td[1]/div[2]/a'
  driver.load(url, wait_xpath)
  soup = driver.get_soup()
  
  pages = soup('ul', class_='pagination')[0]('a')
  num_pages = len(pages)
  products = set()
  for i in range(num_pages):
      element_xpath = '//*[@id="top-nav"]/div/ul/li[%d]/a' % (2 + i)
      driver.click(element_xpath, wait_xpath)
      soup = driver.get_soup()
      for tag in soup('td', class_='first-cell'):
          product = tag.find('div', class_='anchor')
          if not product:
              continue
          else:
              if product.find('a'):
                  products.add(product.find('a')['href'])
  </code></pre>
  
  <p><strong>UPDATE</strong></p>
  
  <p>Part of my problem is that I was reloading the first page and expecting it to change. But even then, moving the clicking line and soup line down after the for-loop, there would be times it would take too long to change.</p>