Using Python and BeautifulSoup (Saved webpage source codes into a local file)
Question: Could you help me with this problem, please?

(Environment: Python 2.7 + BeautifulSoup 4.3.2)

I am trying to using Python and BeautifulSoup to pick up information on a
webpage. Because the webpage is in the company website requires login and
redirection, so I copy the source codes of the target page into a file and
save it as “example.html” in C:\ for the convenience of practicing.

This the a part of the original codes:

    
    
    <tr class="ghj">
        <td><span class="city-sh"><sh src="./citys/1.jpg" alt="boy" title="boy" /></span><a href="./membercity.php?mode=view&amp;u=12563">port_new_cape</a></td>
        <td class="position"><a href="./search.php?id=12563&amp;sr=positions" title="Search positions">452</a></td>
        <td class="details"><div>South</div></td>
        <td>May 09, 1997</td>
        <td>Jan 23, 2009 12:05 pm&nbsp;</td>
    </tr>
    

The codes so far I worked out is:

    
    
    from bs4 import BeautifulSoup
    import re
    import urllib2
    
    url = "C:\example.html"
    page = urllib2.urlopen(url)
    soup = BeautifulSoup(page.read())
    
    cities = soup.find_all('span', {'class' : 'city-sh'})
    
    for city in cities:
    print city
    

** this is just the first stage of testing so somewhat not completed.

However when I run it, it gives error message, seems it’s improper to use
“urllib2.urlopen” to open a local file.

Traceback (most recent call last): File "C:\Python27\Testing.py", line 8, in
page = urllib2.urlopen(url) File "C:\Python27\lib\urllib2.py", line 127, in
urlopen return _opener.open(url, data, timeout) File
"C:\Python27\lib\urllib2.py", line 404, in open response = self._open(req,
data) File "C:\Python27\lib\urllib2.py", line 427, in _open 'unknown_open',
req) File "C:\Python27\lib\urllib2.py", line 382, in _call_chain result =
func(*args) File "C:\Python27\lib\urllib2.py", line 1247, in unknown_open
raise URLError('unknown url type: %s' % type) URLError:

So could you please teach me, in what way, I can practice by using a local
file? Thank you.


Answer: The best way to open a local file with BeautifulSoup is to pass it an open
file handler directly.
<http://www.crummy.com/software/BeautifulSoup/bs4/doc/#making-the-soup>

    
    
    from bs4 import BeautifulSoup
    
    soup = BeautifulSoup(open("C:\\example.html"))
    
    for city in soup.find_all('span', {'class' : 'city-sh'}):
        print(city)
    

