title: StaleElementReferenceException in Python
body: <p>I am trying to scrape data from the Sunshine List website (<a href="http://www.sunshinelist.ca/" rel="nofollow noreferrer">http://www.sunshinelist.ca/</a>) using the Selenium package but I get the following error mentioned  below. From several other related posts I understand that I need to use the WebDriverWait to explicitly ask the driver to wait/refresh but I am unable to identify where and how I should call the function. </p>
  
  <p><a href="https://i.stack.imgur.com/CaCn2.jpg" rel="nofollow noreferrer">Screenshot of Error</a></p>
  
  <blockquote>
    <p>StaleElementReferenceException: Message: The element reference
    of (tr class="even") stale: either the element is no longer attached to the DOM or the
    page has been refreshed</p>
  </blockquote>
  
  <pre><code>import numpy as np
  import pandas as pd
  import requests
  import time
  from bs4 import BeautifulSoup
  from selenium import webdriver
  from selenium.common.exceptions import NoSuchElementException
  from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
  from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
  from selenium.webdriver.common.keys import Keys
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.common.exceptions import StaleElementReferenceException
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.webdriver.common.by import By
  
  ffx_bin = FirefoxBinary(r'C:\Users\BhagatM\AppData\Local\Mozilla Firefox\firefox.exe')
  ffx_caps = DesiredCapabilities.FIREFOX
  ffx_caps['marionette'] = True
  driver = webdriver.Firefox(capabilities=ffx_caps,firefox_binary=ffx_bin)
  driver.get("http://www.sunshinelist.ca/")
  driver.maximize_window()
  
  tablewotags1=[]
  
  while True:
      divs = driver.find_element_by_id('datatable-disclosures')
      divs1=divs.find_elements_by_tag_name('tbody')
  
      for d1 in divs1:
          div2=d1.find_elements_by_tag_name('tr')
          for d2 in div2:
              tablewotags1.append(d2.text)
  
      try:
          driver.find_element_by_link_text('Next →').click()
      except NoSuchElementException:
          break
  
  year1=tablewotags1[0::10]
  name1=tablewotags1[3::10]
  position1=tablewotags1[4::10]
  employer1=tablewotags1[1::10]  
  
  
  df1=pd.DataFrame({'Year':year1,'Name':name1,'Position':position1,'Employer':employer1})
  df1.to_csv('Sunshine List-1.csv', index=False)
  </code></pre>