Difference between URLLIB2 call in IDLE and from Django?
Question: The following piece of code works as expected when running in a local install
of django apache 2.2

    
    
     fx = urllib2.Request(f);
     fx.add_header('User-Agent','Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.36 Safari/525.19');
     url_opened = urllib2.urlopen(fx);
    

However when I enter that code into IDLE on the same machine I get the
following error:

    
    
        url_opened = urllib2.urlopen(fx);
      File "C:\Python25\lib\urllib2.py", line 124, in urlopen
        return _opener.open(url, data)
      File "C:\Python25\lib\urllib2.py", line 387, in open
        response = meth(req, response)
      File "C:\Python25\lib\urllib2.py", line 498, in http_response
        'http', request, response, code, msg, hdrs)
      File "C:\Python25\lib\urllib2.py", line 425, in error
        return self._call_chain(*args)
      File "C:\Python25\lib\urllib2.py", line 360, in _call_chain
        result = func(*args)
      File "C:\Python25\lib\urllib2.py", line 506, in http_error_default
        raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
    HTTPError: HTTP Error 407: Proxy Authentication Required
    

Any ideas?


Answer: `urllib` and `urllib2` I think look at environment variables for proxies if
one isn't set programatically. Maybe the proxy environment variables haven't
been set properly in IDLE?

Compare the output of the following from IDLE to the Django program:

    
    
    import os, pprint
    for k in os.environ:
        if 'proxy' in k.lower(): # look for proxy environment variables
            print k, os.environ[k]
    

**EDIT:** Quoting
<http://docs.python.org/library/urllib2.html#urllib2.ProxyHandler>:

    
    
    Cause requests to go through a proxy. If proxies is given, it must be a 
    dictionary mapping protocol names to URLs of proxies. The default is to read the 
    list of proxies from the environment variables. If no proxy environment 
    variables are set, in a Windows environment, proxy settings are obtained from 
    the Internet Settings section and in a Mac OS X environment, proxy 
    information is retrieved from the OS X System Configuration Framework.
    
    To disable autodetected proxy pass an empty dictionary.

Maybe Django creates a `ProxyHandler`? Try calling `urllib2.ProxyHandler()` in
IDLE.

