How to fix "unexpected keyword argument 'useChardet'" in html5lib
Question: I'm using html5lib and after updating it to the latest version, I keep getting
this error:

    
    
    Traceback (most recent call last):
      File "/home/travis/build/freelawproject/juriscraper/tests/test_everything.py", line 119, in test_scrape_all_example_files
        site.parse()
      File "/home/travis/build/freelawproject/juriscraper/juriscraper/AbstractSite.py", line 95, in parse
        self.html = self._download()
      File "/home/travis/build/freelawproject/juriscraper/juriscraper/AbstractSite.py", line 384, in _download
        html_tree = self._make_html_tree(text)
      File "/home/travis/build/freelawproject/juriscraper/juriscraper/opinions/united_states/federal_appellate/ca11_u.py", line 26, in _make_html_tree
        e = html5parser.document_fromstring(text)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/lxml/html/html5parser.py", line 64, in document_fromstring
        return parser.parse(html, useChardet=guess_charset).getroot()
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/html5parser.py", line 235, in parse
        self._parse(stream, False, None, *args, **kwargs)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/html5parser.py", line 85, in _parse
        self.tokenizer = _tokenizer.HTMLTokenizer(stream, parser=self, **kwargs)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/_tokenizer.py", line 36, in __init__
        self.stream = HTMLInputStream(stream, **kwargs)
      File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/html5lib/_inputstream.py", line 149, in HTMLInputStream
        return HTMLUnicodeInputStream(source, **kwargs)
    TypeError: __init__() got an unexpected keyword argument 'useChardet'
    

The code I'm using is very simple:

    
    
    from lxml.html import html5parser
    html5parser.document_fromstring(u'<html></html')
    

Any ideas?


Answer: Turns out that if you feed a unicode object to the `document_fromstring`
method, it barfs. It didn't used to because this only happened when I updated
my dependencies.

Anyway, the fix is easy:

    
    
    html5parser.document_fromstring(u'<html></html'.encode('utf-8'))
    

