title: python - Find email address on page with selenium
body: <p>I am trying to get a list of email addresses from a website and am very close. The code I have can be seen below. I am getting the following error.</p>
  
  <p>What happens is that there is a page of links which are then clicked on and in the following page there is an email address.</p>
  
  <p>I am trying to print out the email address inside each of the pages after the link is clicked.</p>
  
  <p><a href="http://www.sportbirmingham.org/directory/sport-birmingham" rel="nofollow noreferrer">Here is an example of a page that the link clicks through to</a>.</p>
  
  <blockquote>
    <p>xTraceback (most recent call last):   File "scrape.py", line 34, in
    
        lookup(driver)   File "scrape.py", line 26, in lookup
        emailAdress = driver.find_element_by_xpath('//div[@id="widget-contact"]//a‌​').get_attribute('hr‌​ef')
    File
    "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
    line 293, in find_element_by_xpath
        return self.find_element(by=By.XPATH, value=xpath)   File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
    line 752, in find_element
        'value': value})['value']   File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
    line 236, in execute
        self.error_handler.check_response(response)   File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py",
    line 192, in check_response
        raise exception_class(message, screen, stacktrace) selenium.common.exceptions.InvalidSelectorException: </p>
  </blockquote>
  
  <p>I am using <code>python 2.7.13</code>.</p>
  
  <pre><code># -*- coding: utf-8 -*-
  
  from lxml import html
  import requests
  import time
  from selenium import webdriver
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.support import expected_conditions as EC
  from selenium.common.exceptions import TimeoutException
  
  def init_driver():
      driver = webdriver.Firefox()
      driver.wait = WebDriverWait(driver, 5)
      return driver
  
  
  def lookup(driver):
      driver.get("http://www.sportbirmingham.org/directory?sport=&amp;radius=15&amp;postcode=B16+8QG&amp;submit=Search")
      try:
          for link in driver.find_elements_by_xpath('//h2[@class="heading"]/a'):
              link.click()
              emailAdress = driver.find_element_by_xpath('//div[@id="widget-contact"]//a‌​').get_attribute('hr‌​ef')
              print emailAdress
      except TimeoutException:
          print "not found"
  
  
  if __name__ == "__main__":
      driver = init_driver()
      lookup(driver)
      time.sleep(5)
      driver.quit()
  </code></pre>
  
  <p>When I try and continue to the next page of links I get the following error</p>
  
  <blockquote>
    <p>File "scrape.py", line 43, in 
        lookup(driver)   File "scrape.py", line 26, in lookup
        links.extend([link.get_attribute('href') for link in driver.find_elements_by_xpath('//h2[@class="heading"]/a')])   File
    "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webelement.py",
    line 139, in get_attribute
        self, name)   File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
    line 465, in execute_script
        'args': converted_args})['value']   File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py",
    line 236, in execute
        self.error_handler.check_response(response)   File "/usr/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py",
    line 192, in check_response
        raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message:
    The element reference is stale. Either the element is no longer
    attached to the DOM or the page has been refreshed.</p>
  </blockquote>