When attempting to upload a UTF-8 text file to Google Drive with the Google API client for python, I get a UnicodeDecodeError
Question: **This question is still unsolved! Please answer if you know**

**Bug**

I have filed a bug here

[http://code.google.com/p/google-api-python-
client/issues/detail?id=131&thanks=131&ts=1335708962](http://code.google.com/p/google-
api-python-client/issues/detail?id=131&thanks=131&ts=1335708962)

While working on my [gdrive-cli](http://github.com/tom-dignan/gdrive-cli)
project, I ran into this error attempting to upload a UTF-8 markdown file,
using the "text/plain" mime-type. I also tried with "text/plain;charset=utf-8"
and got the same result.

Here is the stacktrace I got:

    
    
    Traceback (most recent call last):
      File "./gdrive-cli", line 155, in <module>
        handle_args(args)
      File "./gdrive-cli", line 92, in handle_args
        handle_insert(args.insert)
      File "./gdrive-cli", line 126, in handle_insert
        filename)
      File "/home/tom/Github/gdrive-cli/gdrive/gdrive.py", line 146, in insert_file
        media_body=media_body).execute()
      File "/usr/local/lib/python2.7/dist-packages/apiclient/http.py", line 393, in execute
        headers=self.headers)
      File "/usr/local/lib/python2.7/dist-packages/oauth2client/client.py", line 401, in new_request
        redirections, connection_type)
      File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1544, in request
        (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
      File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1294, in _request
        (response, content) = self._conn_request(conn, request_uri, method, body, headers)
      File "/usr/local/lib/python2.7/dist-packages/httplib2/__init__.py", line 1231, in _conn_request
        conn.request(method, request_uri, body, headers)
      File "/usr/lib/python2.7/httplib.py", line 955, in request
        self._send_request(method, url, body, headers)
      File "/usr/lib/python2.7/httplib.py", line 989, in _send_request
        self.endheaders(body)
      File "/usr/lib/python2.7/httplib.py", line 951, in endheaders
        self._send_output(message_body)
      File "/usr/lib/python2.7/httplib.py", line 809, in _send_output
        msg += message_body
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4518: ordinal not in range(128)
    

And the command I had to issue to generate it was:

    
    
    gdrive-cli --insert README.md "readme file" none "text/plain" README.md
    

You can get the exact README.md file at the time this problem occurred here,
<http://tomdignan.com/files/README.md>

The relevant code from the SDK examples follows. The parameters going in are
in order:

a service instance, "README.md", "readme file", None (python keyword),
"text/plain", and "README.md"

    
    
    def insert_file(service, title, description, parent_id, mime_type, filename):
        """Insert new file.
    
        Args:
            service: Drive API service instance.
            title: Title of the file to insert, including the extension.
            description: Description of the file to insert.
            parent_id: Parent folder's ID.
            mime_type: MIME type of the file to insert.
            filename: Filename of the file to insert.
        Returns:
            Inserted file metadata if successful, None otherwise.
        """
        media_body = MediaFileUpload(filename, mimetype=mime_type)
        body = {
            'title': title,
            'description': description,
            'mimeType': mime_type
        }
    
        # Set the parent folder.
        if parent_id:
            body['parentsCollection'] = [{'id': parent_id}]
    
    
        try:
            file = service.files().insert(
                    body=body,
                    media_body=media_body).execute()
    
            # Uncomment the following line to print the File ID
            # print 'File ID: %s' % file['id']
    
            return file
        except errors.HttpError, error:
            print "TRACEBACK"
            print traceback.format_exc()
            print 'An error occured: %s' % error
            return None
    


Answer: Traceback:

    
    
    ...
    resp = pool.request(method, page, fields = fields)
      File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/request.py", line 79, in request
    **urlopen_kw)
      File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/request.py", line 139, in request_encode_body
    **urlopen_kw)
      File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/connectionpool.py", line 415, in urlopen
    body=body, headers=headers)
      File "/usr/lib/python2.7/site-packages/urllib3-dev-py2.7.egg/urllib3/connectionpool.py", line 267, in _make_request
    conn.request(method, url, **httplib_request_kw)
      File "/usr/lib64/python2.7/httplib.py", line 958, in request
        self._send_request(method, url, body, headers)
      File "/usr/lib64/python2.7/httplib.py", line 992, in _send_request
        self.endheaders(body)
      File "/usr/lib64/python2.7/httplib.py", line 954, in endheaders
        self._send_output(message_body)
      File "/usr/lib64/python2.7/httplib.py", line 812, in _send_output
        msg += message_body
    UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 181: ordinal not in range(128)
    

urllib3

filepost.py:

    
    
    def encode_multipart_formdata(fields, boundary=None):
        ...
        body = BytesIO()
        ...
        return body.getvalue(), content_type
    

request.py:

    
    
    def request(self, method, url, fields=None, headers=None, **urlopen_kw):
        ...
        else:
            return self.request_encode_body(method, url, fields=fields,
                                            headers=headers,
                                            **urlopen_kw)
    
    def request_encode_body(self, method, url, fields=None, headers=None,
                                encode_multipart=True, multipart_boundary=None,
                                **urlopen_kw):
        ...
        if encode_multipart:
            body, content_type = encode_multipart_formdata(fields or {},
                                                           boundary=multipart_boundary)
        ...
        headers = headers or {}
        headers.update({'Content-Type': content_type})
    
        return self.urlopen(method, url, body=body, headers=headers,
                            **urlopen_kw)
    

httplib

httplib.py:

    
    
    def _send_request(self, method, url, body, headers):
        ...
        if body and ('content-length' not in header_names):
            self._set_content_length(body)
        ...
    
    def _set_content_length(self, body):
        # Set the content-length based on the body.
        thelen = None
        try:
            thelen = str(len(body))
        except TypeError, te:
            # If this is a file-like object, try to
            # fstat its file descriptor
            try:
                thelen = str(os.fstat(body.fileno()).st_size)
            except (AttributeError, OSError):
                # Don't send a length if this failed
                if self.debuglevel > 0: print "Cannot stat!!"
    
        if thelen is not None:
            self.putheader('Content-Length', thelen)
    
        ...
    
    def _send_output(self, message_body=None):
        ...
        if isinstance(message_body, str):
            msg += message_body
            message_body = None
        self.send(msg)
        if message_body is not None:
            #message_body was not a string (i.e. it is a file) and
            #we must run the risk of Nagle
            self.send(message_body)
    
        ...
    
    def send(self, data):
        ...
        blocksize = 8192
        if hasattr(data,'read') and not isinstance(data, array):
            if self.debuglevel > 0: print "sendIng a read()able"
            datablock = data.read(blocksize)
            while datablock:
                self.sock.sendall(datablock)
                datablock = data.read(blocksize)
        else:
            self.sock.sendall(data)
    

My script.py:

    
    
    #!/usr/bin/env python
    
    from io import BytesIO
    
    data = BytesIO()
    
    print type(data.getvalue()
    

**$./script.py**

**< type 'str'>**

My solution:

    
    
    #!/usr/bin/python
    
    from urllib3 import HTTPConnectionPool, filepost
    from io import BytesIO, SEEK_SET
    
    pool = HTTPConnectionPool('web')
    
    archive_name = '/var/archives/mock.tar.gz'
    
    fields = {'uploadedfile' : (archive_name, open(archive_name).read())}
    
    value, content_type = filepost.encode_multipart_formdata(fields)
    
    headers = {'Content-Type' : content_type, 'Content-Length' : len(value)}
    
    # or any temporary file. without Content-Length
    
    data = BytesIO()
    
    data.write(value)
    
    data.seek(0, SEEK_SET)
    
    resp = pool.urlopen('POST', '/upload/uploader.php', data, headers)
    
    print resp.status
    

**$./upload.py**

**200**

**/var/www/upload# ls**

**mock.tar.gz upload.html upload.php**

