how to grab a web image which has a dynamic src ID in python
Question: it is not a static url but a address like xxx.xxx.com/xxx/run

the image is dynamically built based on daily status, so I can't grab it using
its URL

is it possible to stimulate a browser and get the whole page contains the
image? if then how?

thanks~


Answer: There's two ways of doing this.

  1. Use something like [Requests](http://docs.python-requests.org/en/latest/) to grab the HTML of the page that your image is on, and then use something like [pyquery](http://pythonhosted.org/pyquery/) to parse the HTML and find the URL of your image. This should work in most cases, except when the URL isn't actually in the page's HTML (i.e. because it's put there by Javascript).

  2. Use something like [Splinter](http://splinter.cobrateam.info/), which lets you programmatically control an actual browser, to grab the URL. This is a bit more of a heavyweight solution, but it runs javascript like a normal browser (because it is).

The first option might look like this: import requests from pyquery import
PyQuery

    
    
    html = requests.get('http://example.com/').text
    html_q = PyQuery(html)
    image_url = html_q('img.my_image_class').attr('src')
    

Whereas the second might look like this:

    
    
    from splinter import Browser
    
    with Browser() as b:
        b.visit('http://example.com/')
        image_url = b.find_by_css('img.my_image_class')['src']
    

Then, just download that URL as you would normally.

* * *

Edit: Here's another example with Requests, this time using a session to store
cookies set by a login form. You'll have to get the URL and the keys for the
data dictionary from the `<form>` and `<input>` elements on the login form;
they won't always be `username` and `password`.

    
    
    import requests
    s = requests.session()
    s.post('https://example.com/dologin', data={'username': 'adam', 'password': 'hunter2'})
    html = s.get('https://example.com/other_page').text
    # and continue as in the first example
    

