BeautifulSoup: Get all product links from specific category
Question: I want to get all the product links from specific category by using
BeautifulSoup in Python.

I have tried the following but don't get a result:

    
    
    import lxml
    import urllib2
    from bs4 import BeautifulSoup
    html=urllib2.urlopen("http://www.bedbathandbeyond.com/store/category/bedding/bedding/quilts-coverlets/12018/1-96?pagSortOpt=DEFAULT-0&view;=grid")
    
    br= BeautifulSoup(html.read(),'lxml')
    
    for links in br.findAll('a', class_='prodImg'):
        print links['href']


Answer: You use urllib2 wrong.

    
    
    import lxml
    import urllib2
    from bs4 import BeautifulSoup
    
    #create a http request
    req=urllib2.Request("http://www.bedbathandbeyond.com/store/category/bedding/bedding/quilts-coverlets/12018/1-96?pagSortOpt=DEFAULT-0&view=grid")
    # send the request
    response = urllib2.urlopen(req)
    # read the content of the response
    html = response.read()
    br= BeautifulSoup(html,'lxml')
    
    for links in br.findAll('a', class_='prodImg'):
        print links['href']
    

