how to make a recursive loop in python
Question: I'm trying to make a web scraper that can cycle though pages using
BeautifulSoup

To do this I am trying to write a function that makes a call to the page I'm
looking for, finds the Href of the next button prints the result, then assigns
it to the request and repeats the function recursively printing each new value
of the next button.

This is what I have and I can't really figure out what it is not working. I
get no error so I think my structure may be off.

Thank you in advance.

    
    
    import urllib.request
    from bs4 import BeautifulSoup
    import re
    
    url = "http://www.calaiswine.co.uk/products/type/all-wines/1.aspx"
    root_url = "http://www.calaiswine.co.uk"
    first_index_url =  '/products/type/all-wines/1.aspx'
    
    htmlFile = urllib.request.urlopen(url);
    
    htmlText = htmlFile.read();
    
    soup = BeautifulSoup(htmlText);
    
    def cycle_to_next_page(foo):
        response = urllib.request.urlopen( root_url + foo)
        soup = BeautifulSoup(response)
        items = [a.attrs.get('href') for a in soup.findAll('a', title='Next')]
        print (cycle_to_next_page(items[0]))
    
    cycle_to_next_page(first_index_url)
    


Answer: **Your recursive function returns nothing** , it just prints.

In Python, a function that doesn't return is considered returning `None`. So,
Python understands your `cycle_to_next_page(first_index_url)` instruction as
if you did:

    
    
    print(print(None))
    

I would personally not use recursion for this example. Just a basic `for` loop
iterating through `items`.

