Python requests_toolbelt MultipartEncoder filename
Question: Using requests_toolbelt to upload large files in a Multipart form, I have
constructed a method below which succeeds in uploading the file, however I
cannot access the posted filename. How do I access the filename on the server?

    
    
    # client-side
    file = open('/Volumes/Extra/test/my_video.mpg', 'rb')
    payload = MultipartEncoder({file.name: file})
    r = requests.post(url, data=payload, headers={'Content-Type': 'application/octet-stream'})
    
    # server-side
    @view_config(route_name='remote.agent_upload', renderer='json')
    def remote_agent_upload(request):
        r = request.response
        fs = request.body_file
        f = open('/Volumes/Extra/tests2/bar.mpg', 'wb')  # wish to use filename here
        f.write(fs.read())
        fs.close()
        f.close()
        return r
    


Answer: OK, it looks like you are using the name of the file as the field name. Also,
the way that you are doing it, seems like the entire post content is being
written to file... Is this the desired outcome? Have you tried to actually
play your mpg files after you write them on the server side?

I don't have an HTTP server readily available to test at the moment which
automagically gives me a request object, but I am assuming that the request
object is a webob.Request object (at least it seems like that is the case,
please correct me if I'm wrong)

OK, let me show you my test. (This works on python3.4, not sure what version
of Python you are using, but I think it should also work on Python 2.7 - not
tested though)

The code in this test is a bit long, but it is heavily commented to help you
understand what I did every step of the way. Hopefully, it will give you a
better understanding of how HTTP requests and responses work in python with
the tools you are using

    
    
    # My Imports
    from requests_toolbelt import MultipartEncoder
    from webob import Request
    import io
    
    # Create a buffer object that can be read by the MultipartEncoder class
    # This works just like an open file object
    file = io.BytesIO()
    
    # The file content will be simple for my test.
    # But you could just as easily have a multi-megabyte mpg file
    # Write the contents to the file
    file.write(b'test mpg content')
    
    # Then seek to the beginning of the file so that the
    # MultipartEncoder can read it from the beginning
    file.seek(0)
    
    # Create the payload
    payload = MultipartEncoder(
        {
    
            # The name of the file upload field... Not the file name
            'uploadedFile': (
    
                # This would be the name of the file
                'This is my file.mpg',
    
                # The file handle that is ready to be read from
                file,
    
                # The content type of the file
                'application/octet-stream'
            )
        }
    )
    
    # To send the file, you would use the requests.post method
    # But the content type is not application-octet-stream
    # The content type is multipart/form-data; with a boundary string
    # Without the proper header type, your server would not be able to
    # figure out where the file begins and ends and would think the
    # entire post content is the file, which it is not. The post content
    # might even contain multiple files
    # So, to send your file, you would use:
    #
    # response = requests.post(url, data=payload, headers={'Content-Type': payload.content_type})
    
    # Instead of sending the payload to the server,
    # I am just going to grab the output as it would be sent
    # This is because I don't have a server, but I can easily
    # re-create the object using this output
    postData = payload.to_string()
    
    # Create an input buffer object
    # This will be read by our server (our webob.Request object)
    inputBuffer = io.BytesIO()
    
    # Write the post data to the input buffer so that the webob.Request object can read it
    inputBuffer.write(postData)
    
    # And, once again, seek to 0
    inputBuffer.seek(0)
    
    # Create an error buffer so that errors can be written to it if there are any
    errorBuffer = io.BytesIO()
    
    # Setup our wsgi environment just like the server would give us
    environment = {
        'HTTP_HOST': 'localhost:80',
        'PATH_INFO': '/index.py',
        'QUERY_STRING': '',
        'REQUEST_METHOD': 'POST',
        'SCRIPT_NAME': '',
        'SERVER_NAME': 'localhost',
        'SERVER_PORT': '80',
        'SERVER_PROTOCOL': 'HTTP/1.0',
        'CONTENT_TYPE': payload.content_type,
        'wsgi.errors': errorBuffer,
        'wsgi.input': inputBuffer,
        'wsgi.multiprocess': False,
        'wsgi.multithread': False,
        'wsgi.run_once': False,
        'wsgi.url_scheme': 'http',
        'wsgi.version': (1, 0)
    }
    
    # Create our request object
    # This is the same as your request object and should have all our info for reading
    # the file content as well as the file name
    request = Request(environment)
    
    # At this point, the request object is the same as what you get on your server
    # So, from this point on, you can use the following code to get
    # your actual file content as well as your file name from the object
    
    # Our uploaded file is in the POST. And the POST field name is 'uploadedFile'
    # Grab our file so that it can be read
    uploadedFile = request.POST['uploadedFile']
    
    # To read our content, you can use uploadedFile.file.read()
    print(uploadedFile.file.read())
    
    # And to get the file name, you can use uploadedFile.filename
    print(uploadedFile.filename)
    

So, I think this modified code will work for you. (Hopefully) Again, not
tested because I don't actually have a server to test with. And also, I don't
know what kind of object your "request" object is on the server side.... OK,
here goes:

    
    
    # client-side
    import requests
    file = open('/Volumes/Extra/test/my_video.mpg', 'rb')
    payload = MultipartEncoder({'uploadedFile': (file.name, file, 'application/octet-stream')})
    r = requests.post('http://somewhere/somefile.py', data=payload, headers={'Content-Type': payload.content_type})
    
    # server-side
    @view_config(route_name='remote.agent_upload', renderer='json')
    def remote_agent_upload(request):
    
        # Write your actual file contents, not the post data which contains multi part boundary
        uploadedFile = request.POST['uploadedFile']
        fs = uploadedFile.file
    
        # The file name is insecure. What if the file name comes through as '../../../etc/passwd'
        # If you don't secure this, you've just wiped your /etc/passwd file and your server is toast
        # (assuming the web user has write permission to the /etc/passwd file
        # which it shouldn't, but just giving you a worst case scenario)
        fileName = uploadedFile.filename
    
        # Secure the fileName here... 
        # Make sure it doesn't have any slashes or double dots, or illegal characters, etc.
        # I'll leave that up to you
    
        # Write the file
        f = open('/Volumes/Extra/tests2/' + fileName, 'wb')
        f.write(fs.read())
    

