Python 2.7 reading and writing "éèàçê" from utf-8 file
Question: I made this script which removes every trailing whitespace characters and
replace all bad french characters by the right ones.

Removing the trailing whitespace characters works but not the part about
replacing the french characters.

The file to read/write are encoded in UTF-8 so I added the utf-8 declaration
above my script but in the end every bad characters (like \u00e9) are being
replaced by litte square.

Any idea why?

script :

    
    
    # --*-- encoding: utf-8 --*--
    
    import fileinput
    import sys
    
    CRLF = "\r\n"
    
    ACCENT_AIGU = "\\u00e9"
    ACCENT_GRAVE = "\\u00e8"
    C_CEDILLE = "\\u00e7"
    A_ACCENTUE = "\\u00e0"
    E_CIRCONFLEXE = "\\u00ea"
    
    CURRENT_ENCODING = "utf-8"
    #Getting filepath
    print "Veuillez entrer le chemin du fichier (utiliser des \\ ou /, c'est pareil) :"
    path = str(raw_input())
    path.replace("\\", "/")
    
    #removing trailing whitespace characters
    for line in fileinput.FileInput(path, inplace=1):
        if line != CRLF:
            line = line.rstrip()
            print line
            print >>sys.stderr, line
        else:
            print CRLF
            print >>sys.stderr, CRLF
    fileinput.close()
    
    #Replacing bad wharacters
    for line in fileinput.FileInput(path, inplace=1):
            line = line.decode(CURRENT_ENCODING)
            line = line.replace(ACCENT_AIGU, "é")
            line = line.replace(ACCENT_GRAVE, "è")
            line = line.replace(A_ACCENTUE, "à")
            line = line.replace(E_CIRCONFLEXE, "ê")
            line = line.replace(C_CEDILLE, "ç")
            line.encode(CURRENT_ENCODING)
            sys.stdout.write(line) #avoid CRLF added by print
            print >>sys.stderr, line
    fileinput.close()
    

# EDIT

the input file contains this type of text :

    
    
     * Cette m\u00e9thode permet d'appeller le service du module de tourn\u00e9e
     * <code>rechercherTechnicien</code> et retourne la liste repr\u00e9sentant le num\u00e9ro 
     * de la tourn\u00e9e ainsi que le nom et le pr\u00e9nom du technicien et la dur\u00e9e 
     * th\u00e9orique por se rendre au point d'intervention.
     * 
    

# EDIT2

Final code if someone is interested, the first part replaces the badly encoded
caracters, the second part removes all right trailing whitespaces caracters.

    
    
    # --*-- encoding: iso-8859-1 --*--
    
    import fileinput
    import re
    
    CRLF = "\r\n"
    
    print "Veuillez entrer le chemin du fichier (utiliser des \\ ou /, c'est pareil) :"
    path = str(raw_input())
    path = path.replace("\\", "/")
    
    def unicodize(seg):
        if re.match(r'\\u[0-9a-f]{4}', seg):
            return seg.decode('unicode-escape')
        return seg.decode('utf-8')
    
    print "Replacing caracter badly encoded"
    with open(path,"r") as f:
        content = f.read()
    replaced = (unicodize(seg) for seg in re.split(r'(\\u[0-9a-f]{4})',content))
    
    with open(path, "w") as o:
        o.write(''.join(replaced).encode("utf-8"))
    
    print "Removing trailing whitespaces caracters"
    for line in fileinput.FileInput(path, inplace=1):
        if line != CRLF:
            line = line.rstrip()
            print line
        else:
            print CRLF
    fileinput.close()
    
    print "Done!"
    


Answer: You are looking for `s.decode('unicode_escape')`:

    
    
    >>> s = r"""
    ...  * Cette m\u00e9thode permet d'appeller le service du module de tourn\u00e9e
    ...  * <code>rechercherTechnicien</code> et retourne la liste repr\u00e9sentant le num\u00e9ro
    ...  * de la tourn\u00e9e ainsi que le nom et le pr\u00e9nom du technicien et la dur\u00e9e
    ...  * th\u00e9orique por se rendre au point d'intervention.
    ...  *
    ... """
    >>> print(s.decode('unicode_escape'))
    
     * Cette méthode permet d'appeller le service du module de tournée
     * <code>rechercherTechnicien</code> et retourne la liste représentant le numéro
     * de la tournée ainsi que le nom et le prénom du technicien et la durée
     * théorique por se rendre au point d'intervention.
     *
    

And don't forget to `encode` your string before writing it to a file (e.g. as
UTF-8):

    
    
    writable_s = s.decode('unicode_escape').encode('utf-8')
    

