title: Python: urlretrieve PDF downloading
body: <p>I am using urllib's urlretrieve() function in Python in order to try to grab some pdf's from websites. It has (at least for me) stopped working and is downloading damaged data (15 KB instead of 164 KB).</p>
  
  <p>I have tested this with several pdf's, all with no success (ie <a href="http://www.mathworks.com/moler/random.pdf/" rel="nofollow">random.pdf</a>). I can't seem to get it to work, and I need to be able to download pdf's for the project I am working on.</p>
  
  <p>Here is an example of the kind of code I am using to download the pdf's (and parse the text using <a href="http://www.foolabs.com/xpdf/download.html/" rel="nofollow">pdftotext.exe</a>):</p>
  
  <pre><code>def get_html(url): # gets html of page from Internet
      import os
      import urllib2
      import urllib
      from subprocess import call
      f_name = url.split('/')[-2] # get file name (url must end with '/')
      try:
          if f_name.split('.')[-1] == 'pdf': # file type
              urllib.urlretrieve(url, os.getcwd() + '\\' + f_name)
              call([os.getcwd() + '\\pdftotext.exe', os.getcwd() + '\\' + f_name]) # use xpdf to output .txt file
              return open(os.getcwd() + '\\' + f_name.split('.')[0] + '.txt').read()
          else:
              return urllib2.urlopen(url).read()
      except:
          print 'bad link: ' + url    
          return ""
  </code></pre>
  
  <p>I am a novice programmer, so any input would be great! Thanks</p>