Cannot bring any url in browser from PyCharm/selenium: http proxy responds regardless of settings
Question: I have Python 2.7.4 and selenium bindings (installed via "pip install
selenium") on Ubuntu 13.04 with Firefox 25. I have PyCharm Community Edition
3.0.1 I'm behind a proxy. I have a very simple python test, test_selenium.py:

    
    
    from selenium import webdriver
    browser = webdriver.Firefox()
    browser.get("seleniumhq.org/")
    

When I try to run the above from PyCharm, a Firefox browser is brought up but
no url shows up in the location bar. Instead the following is displayed:

    
    
    Traceback (most recent call last):
      File "/home/nimbula/svn/nimbula/ui-selenium-tests/test-selenium.py", line 18, in <module>
        browser = webdriver.Firefox()
      File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 60, in __init__
        desired_capabilities=capabilities)
      File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 71, in __init__
        self.start_session(desired_capabilities, browser_profile)
      File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 113, in start_session
        'desiredCapabilities': desired_capabilities,
      File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 164, in execute
        self.error_handler.check_response(response)
      File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 136, in check_response
        raise exception_class(value)
    

selenium.common.exceptions.WebDriverException: Message: u'

The message is an html file that contains the following error:

    
    
    The proxy could not connect to the destination in time. Please verify the site you are attempting to access and retry.\n    </td>\n  </tr>\n</table>\n<!--/Content-->\n\n<!--Info-->\n<table class="infoTable">\n  <tr>\n    <td class="infoData">\n      <b>URL: </b><script type="text/javascript">break_line("http://127.0.0.1:51991/hub/session");</script>
    

Thus from the above it looks like selenium webdriver is trying to access
127.0.0.1:5199 but the proxy intercepts the call. I tried the following code:

    
    
    from selenium import webdriver
    
    PROXY_HOST = "<corp proxy>"
    PROXY_PORT = 80
    
    profile = webdriver.FirefoxProfile()
    profile.set_preference("network.proxy.type", 1)
    profile.set_preference("network.proxy.http", PROXY_HOST)
    profile.set_preference("network.proxy.http_port", PROXY_PORT)
    profile.set_preference("network.proxy.ftp", PROXY_HOST)
    profile.set_preference("network.proxy.ftp_port", PROXY_PORT)
    profile.set_preference("network.proxy.ssl", PROXY_HOST)
    profile.set_preference("network.proxy.ssl_port", PROXY_PORT)
    profile.set_preference("network.proxy.socks", PROXY_HOST)
    profile.set_preference("network.proxy.socks_port", PROXY_PORT)
    profile.set_preference("network.proxy.no_proxies_on", "127.0.0.1")
    profile.update_preferences()
    
    browser = webdriver.Firefox(firefox_profile=profile)
    browser.get("seleniumhq.org/")
    

Still the same error is displayed. Since I have the browser window open, I
checked Networking settings and they are as specified in the python code: the
127.0.0.1 shows up in the "No proxy for" box. I also tried the following:

  1. Set proxy to None in PyCharm -> Settings -> Http Proxy, execute code that does not set proxies.

  2. Set proxy to "Use proxy" in PyCharm -> Settings -> Http Proxy, and specify all the required information with 127.0.0.1 in exceptions box.

None of the above worked. I also tried executing my script from command line.
It only works if I first execute "unset http_proxy" and then execute the
script without proxy settings. If I try with proxy settings, it does not work
again. I need to be able to run the tests from PyCharm. Thus my question is
two fold:

  1. How do I force no proxy from PyCharm (the Settings->Proxy->None didn't work)? Perhaps, if I run "unset http_proxy" on command line and then start PyCharm from that same command line window, it may work. How do I start pycharm from command line? 'pycharm' is not found, /bin/pycharm is not found either.

  2. How do I force Firefox to have "None" for proxy setting from selenium?

Thanks for any inputs.


Answer: Here are my findings so far:

  1. Forcing PyCharm to use no proxy:

a. In a command line window (Accessories - > Terminal) unset proxy:

    
        unset http_proxy
    

b. Start pycharm from the same command line window:

    
        <install dir>/bin/pycharm.sh
    

The above solves my problem.

  2. I cannot find a way to set proxy in Firefox to "No proxy" from selenium/python. When launching webdirver with the following:
    
        browser = webdriver.Firefox(proxy=None)
    

the browser window that's launnched does not have proxy set to "No proxy" but
to "Use system proxy settings". If anyone is aware of a way to set Firefox
proxy to "No proxy" programmatically from python, please share. Otherwise, I
think it is a limitation of selenium/python.

I am not certain how selenium webdriver works, but my guess is that setting
browser's proxy to "No proxy" (if possible) may still not fix the above
problem. It may be necessary to have system proxy unset, in order for
webdriver to work properly. On unix use

    
    
    unset http_proxy
    

on windows

    
    
    set proxy to none from IE (that will change the system proxy).
    

If anybody has an explanation or better solution, please share.

