working with lynx in python terminal +displaying spanish characters not working
Question: I am querying lynx browser with a URL and getting the output from the
terminal, but the problem is the output is from a peruvian website and the
spanish characters are not being displayed correctly on the terminal. I am
passing the -assume_charset and -assume_unrec_charset flags along with my call
to the url so it should replace the character set with the one I specify if
the website does not specify one. And for both these parameters I have tried
Latin 1, Latin 3 and Latin 4 and none of them seem to work. I would like to
know how I can fix this problem. I am using python subprocess module and
passing the lynx query as a parameter to subprocess.Popen(...) and then
reading the output from STDOUT. My code is the following:

    
    
        import subprocess
    def get_urlData(url):
        cmd = "lynx -dump -nolist -notitle -assume_charset =\"ISO-8859-1\" -assume_unrec_charset=\"ISO-8859-1\" "+url
        lynx = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
        file_data = lynx.stdout.read()
        #file_data = file_data.decode('ISO-8859-1','replace')
        return file_data
    

When I store the returned file_data in an object and attempt to print it on
the python terminal, the following is a part of the output I get which I am
including to clarify the problem:

Per\xedodo: Periodo de Gobierno 2006- 2011. Legislatura: Primera\n Legislatura
Ordinaria 2010\n N\xfamero: 04903/2010-CR Fecha Presentaci\u0137n:
07/13/2011\n  
Proponente: Congreso\n  
Grupo Parlamentario: Multipartidario\n  
T\xedtulo: LEY QUE EXTINGUE LA DEUDA TRIBUTARIA PROVENIENTE DE LA\n
IMPORTACI\u0136N Y/O VENTA DE COMBUSTIBLE TURBO A1\n

clearly the character \u0137 needs to be some other spanish character but I
dont know how to get that to display on my terminal. Any help would be
appreciated.


Answer: \u0136 and \u0137 are UTF characters , your charset in cmd is ISO-8859-1...
you must have the same character enconding.

