Scraping specific elements from page
Question: I am new to python, and I was looking into using scrapy to scrape specific
elements on a page.

I need to fetch the Name and phone number listed on a members page.

This script will fetch the entire page, what can I add/change to fetch only
those specific elements?

    
    
        import scrapy
    
    class DmozSpider(scrapy.Spider):
        name = "dmoz"
        allowed_domains = ["fali.org"]
        start_urls = [
            "http://www.fali.org/members/",
        ]
    
        def parse(self, response):
            filename = response.url.split("/?id=")[-2] + '%random%'
            with open(filename, 'wb') as f:
                f.write(response.body)
    


Answer: I cannot see a page: <http://www.fali.org/members/>

instead it redirects to the home page.

That makes it impossible to give specifics.

Here is an example:

    
    
        article_title = response.xpath("//td[@id='HpWelcome']/h2/text()").extract()
    

That parses "Florida Association of Licensed Investigators (FALI)" from their
homepage.You can get browser plugins to help you figure out xpaths. XPath
Helper on chrome makes it easy.

That said -- go through the tutorials posted above. Because you are gonna have
more questions I'm sure and broad questions like this aren't taken well on
stack-overflow.

