Parse content from select menu, Python+BeautifulSoup
Question: I am trying to parse data from a page using python which can be pretty
straightforward but all the data is hidden under jquery elements and such
which makes it harder to grab the data. Please forgive me as i am a newbie to
Python and programming as a whole so still getting familiar with it.The
website i am getting it from is <http://www.asusparts.eu/partfinder/Asus/All>
In One/E Series so i just need all the data from the E This is the code i have
so far:

    
    
    import string, urllib2, csv, urlparse, sys
    from bs4 import BeautifulSoup
    
    changable_url = 'http://www.asusparts.eu/partfinder/Asus/All%20In%20One/E%20Series'
    page = urllib2.urlopen(changable_url)
    base_url = 'http://www.asusparts.eu'
    soup = BeautifulSoup(page)
    
    redirects = []
    model_info = []
    
    select = soup.find(id='myselectListModel')
    print select.get_text()
    
    
    options = select.findAll('option')
    
    for option in options:
        if(option.has_attr('redirectvalue')):
           redirects.append(option['redirectvalue'])
    
    for r in redirects:
        rpage = urllib2.urlopen(base_url + r.replace(' ', '%20'))
        s = BeautifulSoup(rpage)
        print s
    
        sys.exit()
    

However the only problem is, it just prints out the data for the first model
which is Asus->All In One->E Series->ET10B->AC Adapter. The actual HTML page
prints out like the [following...](http://codepad.org/3f4GEdJb) (output was
too long - just pasted the main output needed)

I am unsure on how i would grab the data for all the E Series parts as i
assumed this would grab everything? Also i would appreciate if any answers you
show relate to the current method i am using as this is the way the person in
charge would like it done, Thanks.

**[EDIT]**

This is how i am trying to parse the HTML:

    
    
    for r in redirects:
        rpage = urllib2.urlopen(urljoin(base_url, quote(r)))
        s = BeautifulSoup(rpage)
        print s
    
        data = soup.find(id='accordion')
    
        selection = data.findAll('td')
    
        for s in selections:
            if(selection.has_attr('class', 'ProduktLista')):
                redirects.append(td['class', 'ProduktLista'])
    

This is the error i come up with:

    
    
    Traceback (most recent call last):
      File "C:\asus.py", line 31, in <module>
        selection = data.findAll('td')
    AttributeError: 'NoneType' object has no attribute 'findAll'
    


Answer: You need to remove the `sys.exit()` call you have in your loop:

    
    
    for r in redirects:
        rpage = urllib2.urlopen(base_url + r.replace(' ', '%20'))
        s = BeautifulSoup(rpage)
        print s
    
        # sys.exit()  # remove this line, no need to exit your program
    

You also may want to use `urllib.quote` to properly quote the URLs you get
from the option dropdown; this removes the need to manually replace spaces
with `'%20'`. Use `urlparse.urljoin()` to construct the final URL:

    
    
    from urllib import quote
    from urlparse import 
    
    for r in redirects:
        rpage = urllib2.urlopen(urljoin(base_url, quote(r)))
        s = BeautifulSoup(rpage)
        print s
    

