title: Downloading large files in Python
body: <p>In python 2.7.3, I try to create a script to download a file over the Internet. I use the urllib2 module.</p>
  
  <p>Here, what I have done :</p>
  
  <pre><code>import urllib2
  
  HTTP_client = urllib2.build_opener()
  #### Here I can modify HTTP_client headers
  URL = 'http://www.google.com'
  data = HTTP_client.open(URL)
  with open ('file.txt','wb') as f:
          f.write(data.read())
  </code></pre>
  
  <p>OK. That's work perfectly.</p>
  
  <p>The problem is when I want to save big files (hundreds of MB). I think that when I call the 'open' method, it downloads the file in memory. But, what about large files ? It will not save 1 GB of data in memory !! What happen if i lost connection, all the downloaded part is lost.</p>
  
  <p>How to download large files in Python like wget does ? In wget, it downloads the file 'directly' in hard disk. We can see the file growning up in size.</p>
  
  <p>I'm surprised there is no method 'retrieve' for doing stuff like </p>
  
  <pre><code>HTTP_client.retrieve(URL, 'filetosave.ext')
  </code></pre>