title: How to parse HTML with entities such as &nbsp; using builtin library ElementTree in Python 2 & Python 3?
body: <p>There are times that you want to parse some reasonably well-formed HTML pages, but you are reluctant to introduce extra library dependency such as BeautifulSoup or lxml. So you will probably like to try the builtin ElementTree first, because it is a standard library, it is fast (implemented in C), and it supports much better interface (such as XPATH support) than the basic HTMLParser. Not to mention, <a href="http://shallowsky.com/blog/programming/parsing-html-python.html" rel="nofollow noreferrer">HTMLParser has its own limitations</a>.</p>
  
  <p>ElementTree will work, until it encounters some entities, such as <code>&amp;nbsp;</code>, which are not handled by default.</p>
  
  <pre><code>import xml.etree.ElementTree as ET
  
  html = '''&lt;html&gt;
      &lt;div&gt;Some reasonably well-formed HTML content.&lt;/div&gt;
      &lt;form action="login"&gt;
      &lt;input name="foo" value="bar"/&gt;
      &lt;input name="username"/&gt;&lt;input name="password"/&gt;
  
      &lt;div&gt;It is not unusual to see &amp;nbsp; in an HTML page.&lt;/div&gt;
  
      &lt;/form&gt;&lt;/html&gt;'''
  et = ET.fromstring(html)
  </code></pre>
  
  <p>Run it on Python 2 or Python 3, you will see this error:</p>
  
  <pre><code>xml.etree.ElementTree.ParseError: undefined entity: line 7, column 38
  </code></pre>
  
  <p>There are some Q&amp;A out there, such as <a href="https://stackoverflow.com/questions/7237466/python-elementtree-support-for-parsing-unknown-xml-entities">this one</a> and <a href="https://stackoverflow.com/questions/15178209/parsing-xhtml-using-xml-etree-elementtree">that one</a>. They hint to use <code>ElementTree.XMLParser().parser.UseForeignDTD(True)</code> but I can not get it work in Python 3.3 and Python 3.4.</p>
  
  <pre><code>$ python3.3
  Python 3.3.5 (v3.3.5:62cf4e77f785, Mar  9 2014, 01:12:57) 
  [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
  Type "help", "copyright", "credits" or "license" for more information.
  &gt;&gt;&gt; import xml.etree.ElementTree as ET
  &gt;&gt;&gt; ET.XMLParser().parser
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  AttributeError: 'xml.etree.ElementTree.XMLParser' object has no attribute 'parser'
  &gt;&gt;&gt; 
  </code></pre>