title: urlopen freezes at random, timout is ignored
body: <p>I have an API manager that connects to an URL and grabs some json. Very simple.
  Cut from the method:</p>
  
  <pre><code>req = Request(url)
  socket.setdefaulttimeout(timeout)
  resp = urlopen(req, None, timeout)
  data = resp.read()
  resp.close()
  </code></pre>
  
  <p>It works fine most of the time, but at random intervals it takes 5 s to complete the request. Even when timeout is set to 0.5 or 1.0 or whatever.
  I have logged it very closely so I am 100% sure that the line that takes time is number #3 (ie. resp = urlopen(req, None, timeout)).</p>
  
  <p>Ive tried all solutions Ive found on the topic of timeout decorators and Timers etc.
  (To list some of them:
  <a href="https://stackoverflow.com/questions/8071254/python-urllib2-urlopen-freezes-script-infinitely-even-though-timeout-is-set">Python urllib2.urlopen freezes script infinitely even though timeout is set</a>, 
  <a href="https://stackoverflow.com/questions/4188723/how-can-i-force-urllib2-to-time-out">How can I force urllib2 to time out?</a>, <a href="https://stackoverflow.com/questions/9312286/timing-out-urllib2-urlopen-operation-in-python-2-4">Timing out urllib2 urlopen operation in Python 2.4</a>, <a href="https://stackoverflow.com/questions/2281850/timeout-function-if-it-takes-too-long-to-finish">Timeout function if it takes too long to finish</a>
  )</p>
  
  <p>But nothing works. My impression is that the thread freezes while urlopen does something and when its done it unfreezes and then all the timers and timeouts returns w timeout errors. but the execution time is still more then 5s. </p>
  
  <p>I've found <a href="http://mail.python.org/pipermail/python-bugs-list/2006-February/031871.html" rel="nofollow noreferrer">this</a> old mailing list regarding urllib2 and handling of chunked encoding. So if the problem is still present then the solution might be to write a custom urlopen based on httplib.HTTP and not httplib.HTTPConnection. 
  Another possible solution is to try some multithreading magic....</p>
  
  <p>Both solutions seem to aggresive. And it bugs me that the timeout does not work all the way.</p>
  
  <p>It is very important that the execution time of the script does not exceed 0.5s. Anyone that knows why I am experiencing the freezes or maybe a way to help me?</p>
  
  <p><strong>Update based on accepted answer:</strong>
  I changed the approach and use curl instead. Together w unix timeout it works just as I want. Example code follows:</p>
  
  <pre><code>t_timeout = str(API_TIMEOUT_TIME)
  c_timeout = str(CURL_TIMEOUT_TIME)
  cmd = ['timeout', t_timeout, 'curl', '--max-time', c_timeout, url]
  prc = Popen(cmd, stdout=PIPE, stderr=PIPE)
  response = prc.communicate()
  </code></pre>
  
  <p>Since curl only accepts int as timeout I added timeout. timeout accepts floats.</p>