What is the best way to unconvert sanitized data?
Question: I have a very large set of data (one of stackoverflow's data dumps) which is
completely in raw and sanitized form.

    
    
    For example:  &lt;/p&gt;
    

Is there an already established way to convert the above and similar back to
their original form for readability and usability? A python script or function
call by chance?


Answer: Here is a solution I had to use to get everything working correctly - note
that the HTML parser didn't do everything I wanted with my data-set

# !/usr/bin/python3

    
    
        import html.parser
        import string
        import sys
    
        # Amount of lines to put into a buffer before writing
        BUFFER_SIZE_LINES = 1024
        html_parser = html.parser.HTMLParser()
    
        # Few HTML reserved chars that are not being cleaned up by HTMLParser
        dict = {}
        dict[ '&quot;' ] = '"'
        dict[ '&apos;' ] = "'"
        dict[ '&amp;' ] = '&'
        dict[ '&lt;' ] = '<'
        dict[ '&gt;' ] = '>'
    
        # Process the file
        def ProcessLargeTextFile(fileIn, fileOut):
            r = open(fileIn, "r")
            w = open(fileOut, "w")
            buff = ""
            buffLines = 0
            for lineIn in r:
    
                lineOut = html_parser.unescape(lineIn)
                for key, value in dict.items():
                    lineOut = lineOut.replace(key,value)
    
                buffLines += 1
    
                if buffLines >= BUFFER_SIZE_LINES:
                    w.write(buff)
                    buffLines = 1
                    buff = ""
    
                buff += lineOut + "\n"
    
            w.write(buff)
            r.close()
            w.close()
    
    
        # Now run
        ProcessLargeTextFile(sys.argv[1],sys.argv[2])
    

