How to do a basic query on yahoo search engine using Python without using any yahoo api?
Question: I want to do a basic query on yahoo search engine from a python script using
Beautiful Soup and urllib. I've done the same for Google which was rather easy
but Yahoo is proving to be a bit difficult. A minimal example script of a
query to yahoo search engine would help. Thank you!


Answer: first, avoid `urllib` \- use
[requests](https://github.com/kennethreitz/requests) instead, it's a much
saner interface.

Then, all links in the returned page have the class `yschttl` and an ID
following the scheme `link-1`, `link-2` and so on. That you can use with
beautiful soup:

    
    
    import requests
    from bs4 import BeautifulSoup
    url = "http://search.yahoo.com/search?p=%s"
    query = "python"
    r = requests.get(url % query) 
    soup = BeautifulSoup(r.text)
    soup.find_all(attrs={"class": "yschttl"})
    
    for link in soup.find_all(attrs={"class": "yschttl"}):
        print "%s (%s)" %(link.text, link.get('href'))
    

Gives us

>
>     Python Programming Language – Official Website (http://www.python.org/)
>     Python - Image Results
> (http://images.search.yahoo.com/search/images?_adv_prop=image&va=python)
>     Python (programming language) - Wikipedia, the free encyclopedia
> (http://en.wikipedia.org/wiki/Python_(programming_language))
>  

and more.

