Soup.find_all is only returning Some of the results in Python 3.5.1
Question: I'm trying to get all of the urls for thumbnails from my webpage that have the
class = "thumb", but soup.find_all is only printing the most recent 22 or so.

Here is the Code:

    
    
    import requests
    from bs4 import BeautifulSoup
    r = requests.get("http://rayleighev.deviantart.com/gallery/44021661/Reddit")
    soup = BeautifulSoup(r.content, "html.parser")
    links = soup.find_all("a", {'class' : "thumb"})
    for link in links:
        print(link.get("href"))
    


Answer: I think you meant to ask about _following the pagination_ and grabbing all the
links in a list. Here is the implementation of that idea - use the `offset`
parameter and grab links until there are no more links present incrementing
the `offset` by 24 (number of links per page):

    
    
    import requests
    from bs4 import BeautifulSoup
    
    
    offset = 0
    links = []
    with requests.Session() as session:
        while True:
            r = session.get("http://rayleighev.deviantart.com/gallery/44021661/Reddit?offset=%d" % offset)
            soup = BeautifulSoup(r.content, "html.parser")
            new_links = [link["href"] for link in soup.find_all("a", {'class': "thumb"})]
    
            # no more links - break the loop
            if not new_links:
                break
    
            links.extend(new_links)
            print(len(links))
            offset += 24
    
    print(links)
    

