Unicode error ascii can't encode character
Question: I am trying to import a csv file in order to train my classifier but I keep
receiving this error

    
    
    traceback (most recent call last):
    File "updateClassif.py", line 17, in <module>
    myClassif = NaiveBayesClassifier(fp, format="csv")
      File "C:\Python27\lib\site-packages\textblob\classifiers.py", line 191, in __init__
        super(NLTKClassifier, self).__init__(train_set, feature_extractor, format, **kwargs)
      File "C:\Python27\lib\site-packages\textblob\classifiers.py", line 123, in __init__
        self.train_set = self._read_data(train_set, format)
      File "C:\Python27\lib\site-packages\textblob\classifiers.py", line 143, in _read_data
        return format_class(dataset, **self.format_kwargs).to_iterable()
      File "C:\Python27\lib\site-packages\textblob\formats.py", line 68, in __init__
        self.data = [row for row in reader]
      File "C:\Python27\lib\site-packages\textblob\unicodecsv\__init__.py", line 106, in next
        row = self.reader.next()
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xe6' in position 55: ordinal not in range(128)
    

The CSV file contains 1600000 lines of tweets so I believe some tweets contain
special characters. I have tried saving it using open office as someone
recommended but still the same result. I also tried using latin encoding but
the same result. This is my code :

    
    
    with codecs.open('tr.csv', 'r' ,encoding='latin-1') as fp:
    myClassif = NaiveBayesClassifier(fp, format="csv")
    

This is the code from the library I am using:

    
    
    def __init__(self, csvfile, fieldnames=None, restkey=None, restval=None,
                     dialect='excel', encoding='utf-8', errors='strict', *args,
                     **kwds):
            if fieldnames is not None:
                fieldnames = _stringify_list(fieldnames, encoding)
            csv.DictReader.__init__(self, csvfile, fieldnames, restkey, restval, dialect, *args, **kwds)
            self.reader = UnicodeReader(csvfile, dialect, encoding=encoding,
                                        errors=errors, *args, **kwds)
            if fieldnames is None and not hasattr(csv.DictReader, 'fieldnames'):
                # Python 2.5 fieldnames workaround. (http://bugs.python.org/issue3436)
                reader = UnicodeReader(csvfile, dialect, encoding=encoding, *args, **kwds)
                self.fieldnames = _stringify_list(reader.next(), reader.encoding)
            self.unicode_fieldnames = [_unicodify(f, encoding) for f in
                                       self.fieldnames]
            self.unicode_restkey = _unicodify(restkey, encoding)
    
        def next(self):
            row = csv.DictReader.next(self)
            result = dict((uni_key, row[str_key]) for (str_key, uni_key) in
                          izip(self.fieldnames, self.unicode_fieldnames))
            rest = row.get(self.restkey)
    


Answer: Note that the traceback says _En_ codeError, not DecodeError. It looks like
the NaiveBayesClassifier is expecting ascii. Either make it accept Unicode,
or, if this is OK for your application, replace non-ascii characters with '?'
or something.

