Why won't .png files download with Python script while it works for other image types?
Question: I am trying to search the source code of a webpage, and download various files
from it using Python. This script searches the source code for .jpg files and
downloads them all as expected. However, upon modifying the script (changing
".jpg" to ".png", as shown below), I get the error:

    
    
    Traceback (most recent call last):
    File "img.py", line 19, in <module> urllib.urlretrieve(images[z], "image"+str(z)+".png")
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 91, in urlretrieve
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 237, in retrieve
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 205, in open
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 461, in open_file
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 475, in open_local_file
    IOError: [Errno 2] No such file or directory: '/images/adapt-icon-search.png?1342791397'
    

Here is the script I am using:

    
    
    import urllib
    import urllib2
    import re
    
    print "enter url of site (such as 'dribbble.com')"
    
    url = raw_input()
    fullurl = "http://"+url
    
    src = urllib2.urlopen(fullurl)
    src = src.read()
    
    images = re.findall('src="(.*\.png[^"]*)', src)
    
    z=0
    while z < len(images):
        urllib.urlretrieve(images[z], "image"+str(z)+".png")
        print "done"
        z+=1
    

Insight as to why this script doesn't work for .png files would be much
appreciated. Many thanks in advance.

UPDATE: below is a sample of the source I am wanting to search through:

    
    
    <span rel="tipsy" title="This shot has rebounds." class="rebound-mark has-rebounds">1</span>
                    </a>            
            </div>
        </div>
        <h2>
            <a href="/Dash" class="url" rel="contact" title="Dash"><img alt="Avatar-new" class="photo fn" src="http://dribbble.s3.amazonaws.com/users/107759/avatars/original/avatar-new.png?1339961321" /> Dash</a>
            <a href="/account/pro" class="badge-link">
        <span class="badge badge-pro">Pro</span>
    </a>
        </h2>
    


Answer: So the error you are getting is this:

> IOError: [Errno 2] No such file or directory: '/images/adapt-icon-
> search.png?1342791397'

What's happening is that the web page you're scraping has some PNG references
that do not include the domain name included in the URL. When you try to fetch
them in your `while` loop, it fails because you're only providing the location
on the remote host: `/images/adapt-icon-search.png?1342791397`.

You need to extend your code to detect those kinds of URLs (which are
perfectly legal, and in fact, very common). For the kind you're hitting here,
you'll just need to prepend the matched URL with the host name of the server
(e.g. `http://dribble.com/`).

You will probably also want to handle relative URLs, which also exclude the
hostname, but start without a `/` character. Those will need to be prepended
with the previous page's path, if there was one. So if you were scraping
`http://dribble.com/foo/bar.html`, you'd need to prepend a relative URL with
`http://dribble.com/foo/`.

There's likely a library that will automate handling of non-absolute URLs for
you, perhaps as part of the web-scraping process. I'm afraid I don't know much
about web scraping first hand, but perhaps somebody else can suggest one in a
comment.

