title: How to Select Items in Dropdown in Selenium
body: <p>Firstly, I have been trying to get the dropdown from this web page: <a href="http://solutions.3m.com/wps/portal/3M/en_US/Interconnect/Home/Products/ProductCatalog/Catalog/?PC_Z7_RJH9U5230O73D0ISNF9B3C3SI1000000_nid=RFCNF5FK7WitWK7G49LP38glNZJXPCDXLDbl" rel="nofollow">http://solutions.3m.com/wps/portal/3M/en_US/Interconnect/Home/Products/ProductCatalog/Catalog/?PC_Z7_RJH9U5230O73D0ISNF9B3C3SI1000000_nid=RFCNF5FK7WitWK7G49LP38glNZJXPCDXLDbl</a></p>
  
  <p>This is the code I have:</p>
  
  <pre><code>    import urllib2
      from bs4 import BeautifulSoup
      import re
      from pprint import pprint
  
      from selenium import webdriver
  
      url = 'http://solutions.3m.com/wps/portal/3M/en_US/Interconnect/Home/Products/ProductCatalog/Catalog/?PC_Z7_RJH9U5230O73D0ISNF9B3C3SI1000000_nid=RFCNF5FK7WitWK7G49LP38glNZJXPCDXLDbl'
  
      element_xpath = '//*[@id="Component1"]'
      driver = webdriver.PhantomJS()
      driver.get(url)
      element = driver.find_element_by_xpath(element_xpath)
      element_xpath = '/option[@value="02"]'
      all_options = element.find_elements_by_tag_name("option")
      for option in all_options:
          print("Value is: %s" % option.get_attribute("value"))
          option.click()
      source = driver.page_source.encode('utf-8', 'ignore')
      driver.quit()
  
      source = str(source)
  
      soup = BeautifulSoup(source, 'html.parser')
  
      print soup
  </code></pre>
  
  <p>What prints out is this:</p>
  
  <pre><code>Traceback (most recent call last):
    File "../../../../test.py", line 58, in &lt;module&gt;
  Value is: XX
      main()
    File "../../../../test.py", line 46, in main
      option.click()
    File "/home/eric/dev/octocrawler-env/local/lib/python2.7/site-packages/selenium-2.33.0-py2.7.egg/selenium/webdriver/remote/webelement.py", line 54, in click
      self._execute(Command.CLICK_ELEMENT)
    File "/home/eric/dev/octocrawler-env/local/lib/python2.7/site-packages/selenium-2.33.0-py2.7.egg/selenium/webdriver/remote/webelement.py", line 228, in _execute
      return self._parent.execute(command, params)
    File "/home/eric/dev/octocrawler-env/local/lib/python2.7/site-packages/selenium-2.33.0-py2.7.egg/selenium/webdriver/remote/webdriver.py", line 165, in execute
      self.error_handler.check_response(response)
    File "/home/eric/dev/octocrawler-env/local/lib/python2.7/site-packages/selenium-2.33.0-py2.7.egg/selenium/webdriver/remote/errorhandler.py", line 158, in check_response
      raise exception_class(message, screen, stacktrace)
  selenium.common.exceptions.ElementNotVisibleException: Message: u'{"errorMessage":"Element is not currently visible and may not be manipulated","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:51413","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\\"sessionId\\": \\"30e4fd50-f0e4-11e3-8685-6983e831d856\\", \\"id\\": \\":wdc:1402434863875\\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/30e4fd50-f0e4-11e3-8685-6983e831d856/element/%3Awdc%3A1402434863875/click"}}' ; Screenshot: available via screen
  </code></pre>
  
  <p>And the weirdest most infuriating bit of it all is that sometimes it actually all works out. I have no clue what's going on here.</p>
  
  <hr>
  
  <p><strong>UPDATE</strong></p>
  
  <p>It seems I have no issues with visibility of dropdown forms on other sites, just this one. Is there something that might make the form invisible (and if so, why only 95% of the time)? Could there be an issue loading the page that might cause things not to be visible?</p>