<p>Because regular expressions scare me, I'm trying to find a way to remove all HTML tags and resolve HTML entities from a string in Python.</p>

[{'answer_id': 37496, 'author': 'bernhardrusch', 'author_id': 3056, 'author_profile': 'https://Stackoverflow.com/users/3056', 'pm_score': 2, 'selected': False, 'text': '<p>How about parsing the HTML data and extracting the data with the help of the parser ?</p>\n\n<p>I\'d try something like the author described in <a href="http://diveintopython.net/html_processing/extracting_data.html" rel="nofollow noreferrer">chapter 8.3 in the Dive Into Python book</a> </p>\n'}, {'answer_id': 37502, 'author': 'Grey Panther', 'author_id': 1265, 'author_profile': 'https://Stackoverflow.com/users/1265', 'pm_score': 3, 'selected': False, 'text': '<p>While I agree with Lucas that regular expressions are not all that scary, I still think that you should go with a specialized HTML parser. This is because the HTML standard is hairy enough (especially if you want to parse arbitrarily "HTML" pages taken off the Internet) that you would need to write a lot of code to handle the corner cases. It seems that <a href="http://docs.python.org/lib/module-htmllib.html" rel="nofollow noreferrer">python includes one out of the box</a>. </p>\n\n<p>You should also check out the <a href="http://utidylib.berlios.de/" rel="nofollow noreferrer">python bindings for TidyLib</a> which can clean up broken HTML, making the success rate of any HTML parsing much higher.</p>\n'}, {'answer_id': 37504, 'author': 'Paige Ruten', 'author_id': 813, 'author_profile': 'https://Stackoverflow.com/users/813', 'pm_score': 1, 'selected': False, 'text': '<p>You might need something more complicated than a regular expression. Web pages often have angle brackets that aren\'t part of a tag, like this:</p>\n\n<pre><code> &lt;div&gt;5 &lt; 7&lt;/div&gt;\n</code></pre>\n\n<p>Stripping the tags with regex will return the string "5 " and treat</p>\n\n<pre><code> &lt; 7&lt;/div&gt;\n</code></pre>\n\n<p>as a single tag and strip it out.</p>\n\n<p>I suggest looking for already-written code that does this for you. I did a search and found this: <a href="http://zesty.ca/python/scrape.html" rel="nofollow noreferrer">http://zesty.ca/python/scrape.html</a> It also can resolve HTML entities.</p>\n'}, {'answer_id': 37506, 'author': 'John Millikin', 'author_id': 3560, 'author_profile': 'https://Stackoverflow.com/users/3560', 'pm_score': 4, 'selected': False, 'text': '<p>Use <a href="http://www.crummy.com/software/BeautifulSoup/" rel="noreferrer">BeautifulSoup</a>! It\'s perfect for this, where you have incoming markup of dubious virtue and need to get something reasonable out of it. Just pass in the original text, extract all the string tags, and join them.</p>\n'}, {'answer_id': 37512, 'author': 'Peter Hoffmann', 'author_id': 720, 'author_profile': 'https://Stackoverflow.com/users/720', 'pm_score': 6, 'selected': True, 'text': '<p>Use <a href="http://lxml.de/" rel="nofollow noreferrer">lxml</a> which is the best xml/html library for python.</p>\n\n<pre><code>import lxml.html\nt = lxml.html.fromstring("...")\nt.text_content()\n</code></pre>\n\n<p>And if you just want to sanitize the html look at the lxml.html.clean <a href="http://lxml.de/lxmlhtml.html#cleaning-up-html" rel="nofollow noreferrer">module</a></p>\n'}, {'answer_id': 37604, 'author': 'Carl Meyer', 'author_id': 3207, 'author_profile': 'https://Stackoverflow.com/users/3207', 'pm_score': 0, 'selected': False, 'text': '<p>Regular expressions are not scary, but writing your own regexes to strip HTML is a sure path to madness (and it won\'t work, either).  Follow the path of wisdom, and use one of the many good HTML-parsing libraries.</p>\n\n<p>Lucas\' example is also broken because "sub" is not a method of a Python string.  You\'d have to "import re", then call re.sub(pattern, repl, string).  But that\'s neither here nor there, as the correct answer to your question does not involve writing any regexes.</p>\n'}, {'answer_id': 38646, 'author': 'Lucas Wilson-Richter', 'author_id': 1157, 'author_profile': 'https://Stackoverflow.com/users/1157', 'pm_score': 0, 'selected': False, 'text': "<p>Looking at the amount of sense people are demonstrating in other answers here, I'd say that using a regex probably isn't the best idea for your situation. Go for something tried and tested, and treat my previous answer as a demonstration that regexes need not be <em>that</em> scary.</p>\n"}, {'answer_id': 5144206, 'author': 'code22', 'author_id': 264308, 'author_profile': 'https://Stackoverflow.com/users/264308', 'pm_score': 2, 'selected': False, 'text': '<p>if you use django you might also use \n<a href="http://docs.djangoproject.com/en/dev/ref/templates/builtins/#striptags" rel="nofollow">http://docs.djangoproject.com/en/dev/ref/templates/builtins/#striptags</a>\n;)</p>\n'}]
['https://Stackoverflow.com/questions/37486', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/3971/']