Upload a recording to Google App Engine from Android app
Question: I have created an android app that does basic voice recordings (stored as
.mp4). I want to add a feature where I can send a recording (just one at a
time, no batches needed) to Google App Engine cloud storage. Then, I want to
be able to listen to these recordings on my (very basic) cloud app. I do not
want to use the
[blobstore](https://cloud.google.com/appengine/docs/python/blobstore/) but the
datastore. I have used the datastore before but always just with a cloud app
and python, never with an android app (and my java is shaky at best).
Previously, I just got other data from my appspot website form and sent it to
the datastore using html and python.

In short, my question is, how do I get my recordings from my android app to
the datastore. Code snippets and/or links to documentation would be very
helpful. Also a short explanation of how these things will communicate/work
together would help my brain connect the dots.

Note, this is a personal app just for my own learning and use so I am not
extremely concerned about security, user accounts, scalability etc.

Thanks for your help!


Answer: Datastore is not a good storage option for this use case, as the maximum
entity size is limited to
[1mb](https://cloud.google.com/appengine/docs/python/datastore/#Python_Quotas_and_limits).
GCS is a recommended option for this use case. Also, if you’re not very
comfortable with Java, you can use Python for your GAE backend.

Regarding your question on how to move recordings from Android to GAE GCS /
Datastore, you can use one of these options:

  1. Use [GCS](https://cloud.google.com/storage/docs/json_api/v1/json-api-examples) JSON API via Google APIs Client library to upload directly from Android app. Thats an easy option and there are plenty of samples available. This library can be used for [Datastore](https://cloud.google.com/datastore/docs/apis/v1beta2/) too if you insist on using it.
  2. Use [Cloud Endpoints](https://cloud.google.com/appengine/docs/python/endpoints/) as a proxy. You can submit a file to an endpoint handler, endpoint handler saves the file in GCS and returns a response. As you’re not concerned with security, here’s an example that doesn’t use OAuth:

>
>     import cloudstorage as gcs
>     import endpoints
>     import os
>  
>     from google.appengine.api import app_identity
>     from protorpc import messages
>     from protorpc import message_types
>     from protorpc import remote
>  
>     class Base64File(messages.Message):
>         file = messages.BytesField(1, required=True,
> variant=messages.Variant.BYTES)
>  
>     class ResponseMSG(messages.Message):
>         message = messages.StringField(1)
>  
>     FILE_RESOURCE = endpoints.ResourceContainer(Base64File,
>
> file_name=messages.StringField(2,required=True),
>
> content_type=messages.StringField(3,required=True))
>  
>     @endpoints.api(name='gcsuploadapi', version='v0.1',
>                                           description='Upload a file to
> GCS.')
>     class GCSUploadAPI(remote.Service):
>         @endpoints.method(FILE_RESOURCE, ResponseMSG,
>                             path='upload/{file_name}', http_method='POST',
>                             name='upload.file')
>         def upload_file(self, request):
>             # get app default bucket and prepare filename (project should
> have billing enabled)
>             bucket_name = os.environ.get('BUCKET_NAME',
>
> app_identity.get_default_gcs_bucket_name())
>             bucket = '/' + bucket_name
>             filename = bucket + '/' + request.file_name
>  
>             # create file (request.content_type contains MIME type
> submitted)
>             write_retry_params = gcs.RetryParams(backoff_factor=1.1)
>             gcs_file = gcs.open(filename, 'w',
>                                 content_type=request.content_type,
>                                 retry_params=write_retry_params)
>             gcs_file.write(request.file)
>             gcs_file.close()
>  
>             return ResponseMSG(message="done")
>  
>     app = endpoints.api_server([GCSUploadAPI])
>  

You don’t have to generate client libraries and can submit directly via HTTP
POST. Endpoints only support HTTPS connections and requests are a subject to
same restrictions as other [GAE
requests](https://cloud.google.com/appengine/docs/python/requests#Python_Quotas_and_limits).
The main concern is the request size of 32 megabytes. Taking into account the
binary data submitted to an endpoint should be base64 encoded, it leaves you
with an approximate limit of 23-24mb of binary data. Another limit is a 60
second deadline for a request, it can be resolved if you go with a [modular
approach](https://cloud.google.com/appengine/docs/python/modules/) for your
GAE app and use a backend instance for your endpoint.

That’s probably the hardest option to implement, but the example I provided
should be a good start.

  3. Use Cloud Endpoints to generate a Blobstore API upload url. Blobstore API allows [uploading to GCS](https://cloud.google.com/appengine/docs/python/blobstore/#Python_Using_the_Blobstore_API_with_Google_Cloud_Storage) instead of Blobstore. Thats a rather easy option too.

