Urllib2 Error in pip under Windows
Question: I have some trouble running pip form ActiveState Python 2.7.2 under Windows.
We use a proxy, which might be part of the issue. The proxy is a non-
authenticating proxy. The proxy settings from the system, manually in e.g.
Firefox or with some simple Python code work fine:

This works as expected:

    
    
    urllib.urlopen('http://www.google.com', proxies={'http': 'http://proxy:port'})
    

It gives a response with header information from google.com:

    
    
    <addinfourl at 61539976L whose fp = <socket._fileobject object at 0x00000000042924F8>>
    

With the proxy being set in the `http_proxy` environment variable I run

    
    
    pip install loremipsum
    

I get

    
    
        Downloading/unpacking loremipsum
          Could not fetch URL http://pypi.python.org/simple/loremipsum: <urlopen error [Errno 11004] getaddrinfo
    failed>
          Will skip URL http://pypi.python.org/simple/loremipsum when looking for download links for loremipsum
          Could not fetch URL http://pypi.python.org/simple/: <urlopen error [Errno 11004] getaddrinfo failed>
          Will skip URL http://pypi.python.org/simple/ when looking for download links for loremipsum
          Cannot fetch index base URL http://pypi.python.org/simple/
          Could not fetch URL http://pypi.python.org/simple/loremipsum/: <urlopen error [Errno 11004] getaddrinfo failed>
          Will skip URL http://pypi.python.org/simple/loremipsum/ when looking for download links for loremipsum
          Could not find any downloads that satisfy the requirement loremipsum
        No distributions at all found for loremipsum
        Exception information:
        Traceback (most recent call last):
          File "C:\ActiveState\ActivePython\lib\site-packages\pip\basecommand.py", line 126,  in main
        self.run(options, args)
      File "C:\ActiveState\ActivePython\lib\site-packages\pip\commands\install.py", line 222, in run
        requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
      File "C:\ActiveState\ActivePython\lib\site-packages\pip\req.py", line 954, in prepare_files
        url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
      File "C:\ActiveState\ActivePython\lib\site-packages\pip\index.py", line 152, in find_requirement
        raise DistributionNotFound('No distributions at all found for %s' % req)
    DistributionNotFound: No distributions at all found for loremipsum
    

The error 11004 seems to indicate a name resolution problem, which behind a
proxy should be a problem with proxy access (or ignorance).

I can test a similar setup (http_proxy variable, same proxy, different Python)
under Linux. Running the above command works nicely. Also accessing the URL in
a browser on the Windows machine works (shows a set of egg and zip files).

I walked the `pip` code to find where it misses an ran the code in an
interactive session. I found that in `C:\ActiveState\ActivePython\lib\site-
packages\pip\downloads.py` basically all downloading preparation and action
happens. In `setup()` (line 125) the `ProxyHandler` is prepared and and
`opener` constructed, which is stored in `urllib2` to be used by further
calls. When running interactively I found that adding an entry for a `https`
proxy was needed. I also added the printing of debug information. This gave me
in an interactive run in iPython:

    
    
    In [1]: import urllib2
    
    In [2]: proxy='proxy:port'
    
    In [3]: proxy_support = urllib2.ProxyHandler({"http": proxy, "ftp": proxy, "https": proxy})
    
    In [4]: opener = urllib2.build_opener(proxy_support, urllib2.CacheFTPHandler, urllib2.HTTPHandler(debugle
    vel=1), urllib2.HTTPSHandler(debuglevel=1))
    
    In [5]: urllib2.install_opener(opener)
    

The proxy for `ftp`, `http` and `https` is indeed the same. I aslo checked
with some printouts that all code handling commandline parameters, etc. did
not mess with the proxy. The proxy is stored the same way in `downloads.py` as
simplified above (read from the http_proxy variable).

After figuring out the URL to fetch the package from pip goes to `__call__()`
at line 74.

First a request is constructed using:

    
    
    In [6]: url = urllib2.Request('http://pypi.python.org/simple/loremipsum', headers={'Accept-encoding': 'identity'})
    

then the request is used with `urllib2.urlopen(url)`:

    
    
    In [7]: response = urllib2.urlopen(url)
    send: 'GET http://pypi.python.org/simple/loremipsum HTTP/1.1\r\nHost: pypi.python.org\r\nUser-Agent: Pyth
    on-urllib/2.7\r\nConnection: close\r\nAccept-Encoding: identity\r\n\r\n'
    reply: 'HTTP/1.1 301 Moved Permanently\r\n'
    header: Server: Varnish
    header: Retry-After: 0
    header: Location: https://pypi.python.org/simple/loremipsum
    header: Content-Length: 0
    header: Accept-Ranges: bytes
    header: Date: Tue, 12 Aug 2014 16:41:39 GMT
    header: Via: 1.1 varnish
    header: X-Served-By: cache-fra1234-FRA
    header: X-Cache: MISS
    header: X-Cache-Hits: 0
    header: X-Timer: S1407861699.491394,VS0,VE0
    header: Connection: close
    header: Age: 0
    send: 'CONNECT pypi.python.org:443 HTTP/1.0\r\n'
    send: '\r\n'
    send: 'GET /simple/loremipsum HTTP/1.1\r\nHost: pypi.python.org\r\nUser-Agent: Python-urllib/2.7\r\nConne
    ction: close\r\nAccept-Encoding: identity\r\n\r\n'
    reply: 'HTTP/1.1 301 Moved Permanently\r\n'
    header: Date: Tue, 12 Aug 2014 16:41:40 GMT
    header: Server: nginx/1.6.0
    header: Location: /simple/loremipsum/
    header: Cache-Control: max-age=600, public
    header: Strict-Transport-Security: max-age=31536000; includeSubDomains
    header: Via: 1.1 varnish
    header: Content-Length: 0
    header: Accept-Ranges: bytes
    header: Via: 1.1 varnish
    header: Age: 44282
    header: X-Served-By: cache-iad2135-IAD, cache-fra1231-FRA
    header: X-Cache: MISS, HIT
    header: X-Cache-Hits: 0, 1
    header: X-Timer: S1407861700.831757,VS0,VE0
    header: Connection: close
    send: 'CONNECT pypi.python.org:443 HTTP/1.0\r\n'
    send: '\r\n'
    send: 'GET /simple/loremipsum/ HTTP/1.1\r\nHost: pypi.python.org\r\nUser-Agent: Python-urllib/2.7\r\nConn
    ection: close\r\nAccept-Encoding: identity\r\n\r\n'
    reply: 'HTTP/1.1 200 OK\r\n'
    header: Date: Tue, 12 Aug 2014 16:41:41 GMT
    header: Server: nginx/1.6.0
    header: Content-Type: text/html; charset=utf-8
    header: X-PYPI-LAST-SERIAL: 794358
    header: Cache-Control: max-age=600, public
    header: Strict-Transport-Security: max-age=31536000; includeSubDomains
    header: Via: 1.1 varnish
    header: Content-Length: 913
    header: Accept-Ranges: bytes
    header: Via: 1.1 varnish
    header: Age: 67708
    header: X-Served-By: cache-iad2121-IAD, cache-fra1231-FRA
    header: X-Cache: HIT, HIT
    header: X-Cache-Hits: 1, 1
    header: X-Timer: S1407861701.174694,VS0,VE0
    header: Vary: Accept-Encoding
    header: Connection: close
    

This seems to be an ok answer. I have the very same code in pip and yet it
fails.

What am I missing? Why is my interactive session working and pip isn't?


Answer: You didn't [specify the
proxy](http://pip.readthedocs.org/en/latest/reference/pip.html?highlight=proxy#cmdoption
--proxy) to pip?

EDIT: summary of the comments: proxy was specified, by ENV `HTTP_PROXY`, but
didn't work at first, and works now.

I looked a bit into urllib, and there is code to handle the windows registry
settings by default. There is `getproxies()`, which returns
`getproxies_environment() or getproxies_registry()`, so you _should_ have been
OK already without any modifications to the EMV _or_ the command line.

