Reading a page and parsing it with minidom.parse or minidom.parseString in Python?
Question: I have either of these codes:

    
    
    import urllib
    from xml.dom import minidom
    res = urllib.urlopen('https://www.google.com/webhp#q=apple&start=10')
    dom = minidom.parse(res)
    

which gives me the error `xml.parsers.expat.ExpatError: syntax error: line 1,
column 0`

Or this:

    
    
    import urllib
    from xml.dom import minidom
    res = urllib.urlopen('https://www.google.com/webhp#q=apple&start=10')
    dom = minidom.parseString(res.read())
    

which gives me the same error. `res.read()` reads fine and is a string.

I would like to parse through the code later. How can I do this using
`xml.dom.minidom`?


Answer: The reason you're getting this error is that the page isn't valid XML. It's
HTML 5. The `doctype` right at the top tells you this, even if you ignore the
content type. You can't parse HTML with an XML parser.*

If you want to stick with what's in the stdlib, you can use
[`html.parser`](https://docs.python.org/3/library/html.parser.html) (Python
3.x) / [`HTMLParser`](https://docs.python.org/2.7/library/htmlparser.html)
(2.x).** However, you may want to consider third-party libraries like `lxml`
(which, despite the name, can parse HTML), `html5lib`, or `BeautifulSoup`
(which wraps up a lower-level parser in a really nice interface).

* Well, unless it's XHTML, or the XML output of HTML5, but that's not the case here.

** Do not use `htmllib` unless you're using an old version of Python without a
working `HTMLParser`. This module is deprecated for a reason.

