Scraping on Python
Question: I wanted to get the caption, no. of likes and comments of the recent 10 images
of a particular user. Using below code I am just able to get the latest one.

Code:

    
    
    from selenium import webdriver
    from bs4 import BeautifulSoup
    import json, time, re
    phantomjs_path = r'C:\Users\ravi.janjwadia\Desktop\phantomjs-2.1.1-windows\bin\phantomjs.exe'
    browser = webdriver.PhantomJS(phantomjs_path)
    user = "barackobama"     
    browser.get('https://instagram.com/' + user)
    time.sleep(0.5)
    soup = BeautifulSoup(browser.page_source, 'html.parser')
    script_tag = soup.find('script',text=re.compile('window\._sharedData'))
    shared_data = script_tag.string.partition('=')[-1].strip(' ;')
    result = json.loads(shared_data)
    print(result['entry_data']['ProfilePage'][0]['user']['media']['nodes'][0]['caption'])
    

Result: LAST CALL: Enter for a chance to meet President Obama this summer
before tonight's deadline. → Link in profile.


Answer: In your code below, you're only retrieving the first node (which is the first
image).

    
    
    print(result['entry_data']['ProfilePage'][0]['user']['media']['nodes'][0]['caption'])
    

To get the info of the recent 10 images of the user try this instead.

    
    
    recent_ten_nodes = result['entry_data']['ProfilePage'][0]['user']['media']['nodes'][:10]
    

To only print the captions, number of likes and comments do this.

    
    
    for node in recent_ten_nodes:
        print node['caption']
        print node['likes']['count']
        print node['comments']['count'] 
    

For storing of these values, it's up to you to decide how you want to store
them.

