api/mobile_api.py
def upload(OS_API=""):
    """ Entry point to upload GPS, Accelerometer, Audio, PowerState, Calls Log, Texts Log,
    Survey Response, and debugging files to s3.

    Behavior:
    The Beiwe app is supposed to delete the uploaded file if it receives an html 200 response.
    The API returns a 200 response when the file has A) been successfully handled, B) the file it
    has been sent is empty, C) the file did not decrypt properly.  We encountered problems in
    production with incorrectly encrypted files (as well as Android generating "rList" files
    under unknown circumstances) and the app then uploads them.  When the device receives a 200
    that is its signal to delete the file.
    When a file is undecryptable (this was tracked to a scenario where the device could not
    create/write an AES encryption key) we send a 200 response to stop that device attempting to
    re-upload the data.
    In the event of a single line being undecryptable (can happen due to io errors on the device)
    we drop only that line (and store the erroring line in an attempt to track it down.

    A 400 error means there is something is wrong with the uploaded file or its parameters,
    administrators will be emailed regarding this upload, the event will be logged to the apache
    log.  The app should not delete the file, it should try to upload it again at some point.

    If a 500 error occurs that means there is something wrong server side, administrators will be
    emailed and the event will be logged. The app should not delete the file, it should try to
    upload it again at some point.

    Request format:
    send an http post request to [domain name]/upload, remember to include security
    parameters (see user_authentication for documentation). Provide the contents of the file,
    encrypted (see encryption specification) and properly converted to Base64 encoded text,
    as a request parameter entitled "file".
    Provide the file name in a request parameter entitled "file_name". """

    # Handle these corner cases first because they requires no database input.
    # Crash logs are from truly ancient versions of the android codebase
    # rList are randomly generated by android
    # PersistedInstallation files come from firebase.
    # todo: stop uploading junk files in the app by putting our files into a folder.
    file_name = request.values.get("file_name", None)
    if (
            not bool(file_name)
            or file_name.startswith("rList")
            or file_name.startswith("PersistedInstallation")
            or not contains_valid_extension(file_name)
    ):
        return render_template('blank.html'), 200

    s3_file_location = file_name.replace("_", "/")
    participant = get_session_participant()

    if participant.unregistered:
        # "Unregistered" participants are blocked from uploading further data.
        # If the participant is unregistered, throw away the data file, but
        # return a 200 "OK" status to the phone so the phone decides it can
        # safely delete the file.
        return render_template('blank.html'), 200

    # block duplicate FTPs.  Testing the upload history is too complex
    if FileToProcess.test_file_path_exists(s3_file_location, participant.study.object_id):
        return render_template('blank.html'), 200

    uploaded_file = get_uploaded_file()
    try:
        uploaded_file = decrypt_device_file(uploaded_file, participant)
    except HandledError:
        return render_template('blank.html'), 200
    except DecryptionKeyInvalidError:
        # when the decryption key is invalid the file is lost.  Nothing we can do.
        # record the event, send the device a 200 so it can clear out the file.
        if REPORT_DECRYPTION_KEY_ERRORS:
            tags = {
                "participant": participant.patient_id,
                "operating system": "ios" if "ios" in request.path.lower() else "android",
                "DecryptionKeyError id": str(DecryptionKeyError.objects.last().id),
                "file_name": file_name,
                "bug_report": DECRYPTION_KEY_ADDITIONAL_MESSAGE,
            }
            sentry_client = make_sentry_client(SentryTypes.elastic_beanstalk, tags)
            sentry_client.captureMessage(DECRYPTION_KEY_ERROR_MESSAGE)
        return render_template('blank.html'), 200

    # if uploaded data actually exists, and has a valid extension
    if uploaded_file and file_name and contains_valid_extension(file_name):
        s3_upload(s3_file_location, uploaded_file, participant.study.object_id)

        # race condition: multiple _concurrent_ uploads with same file path. Behavior without
        # try-except is correct, but we don't care about reporting it. Just send the device a 500
        # error so it skips the file, the followup attempt receives 200 code and deletes the file.
        try:
            FileToProcess.append_file_for_processing(
                s3_file_location, participant.study.object_id, participant=participant
            )
        except ValidationError as e:
            # Real error is a second validation inside e.error_dict["s3_file_path"].
            # Ew; just test for this string instead...
            if S3_FILE_PATH_UNIQUE_CONSTRAINT_ERROR in str(e):
                # this tells the device to just move on to the next file, try again later.
                return abort(500)
            else:
                raise

        UploadTracking.objects.create(
            file_path=s3_file_location,
            file_size=len(uploaded_file),
            timestamp=timezone.now(),
            participant=participant,
        )
        return render_template('blank.html'), 200

    elif not uploaded_file:
        # if the file turns out to be empty, delete it, we simply do not care.
        return render_template('blank.html'), 200
    else:
        return make_upload_error_report(participant.patient_id, file_name)
 Entry point to upload GPS, Accelerometer, Audio, PowerState, Calls Log, Texts Log,
    Survey Response, and debugging files to s3.

    Behavior:
    The Beiwe app is supposed to delete the uploaded file if it receives an html 200 response.
    The API returns a 200 response when the file has A) been successfully handled, B) the file it
    has been sent is empty, C) the file did not decrypt properly.  We encountered problems in
    production with incorrectly encrypted files (as well as Android generating "rList" files
    under unknown circumstances) and the app then uploads them.  When the device receives a 200
    that is its signal to delete the file.
    When a file is undecryptable (this was tracked to a scenario where the device could not
    create/write an AES encryption key) we send a 200 response to stop that device attempting to
    re-upload the data.
    In the event of a single line being undecryptable (can happen due to io errors on the device)
    we drop only that line (and store the erroring line in an attempt to track it down.

    A 400 error means there is something is wrong with the uploaded file or its parameters,
    administrators will be emailed regarding this upload, the event will be logged to the apache
    log.  The app should not delete the file, it should try to upload it again at some point.

    If a 500 error occurs that means there is something wrong server side, administrators will be
    emailed and the event will be logged. The app should not delete the file, it should try to
    upload it again at some point.

    Request format:
    send an http post request to [domain name]/upload, remember to include security
    parameters (see user_authentication for documentation). Provide the contents of the file,
    encrypted (see encryption specification) and properly converted to Base64 encoded text,
    as a request parameter entitled "file".
    Provide the file name in a request parameter entitled "file_name". 
Entry point to upload GPS, Accelerometer, Audio, PowerState, Calls Log, Texts Log,
Survey Response, and debugging files to s3.

The Beiwe app is supposed to delete the uploaded file if it receives an html 200 response.
The API returns a 200 response when the file has A) been successfully handled, B) the file it
has been sent is empty, C) the file did not decrypt properly.  We encountered problems in
production with incorrectly encrypted files (as well as Android generating "rList" files
under unknown circumstances) and the app then uploads them.  When the device receives a 200
that is its signal to delete the file.
When a file is undecryptable (this was tracked to a scenario where the device could not
create/write an AES encryption key) we send a 200 response to stop that device attempting to
re-upload the data.
In the event of a single line being undecryptable (can happen due to io errors on the device)
we drop only that line (and store the erroring line in an attempt to track it down.

A 400 error means there is something is wrong with the uploaded file or its parameters,
administrators will be emailed regarding this upload, the event will be logged to the apache
log.  The app should not delete the file, it should try to upload it again at some point.

If a 500 error occurs that means there is something wrong server side, administrators will be
emailed and the event will be logged. The app should not delete the file, it should try to
upload it again at some point.

Request format:
send an http post request to [domain name]/upload, remember to include security
parameters . Provide the contents of the file,
encrypted  and properly converted to Base64 encoded text,
as a request parameter entitled "file".
Provide the file name in a request parameter entitled "file_name".
Entry point to upload GPS Accelerometer Audio PowerState Calls Log Texts Log Survey Response and debugging files to s3 . The Beiwe app is supposed to delete the uploaded file if it receives an html 200 response . The API returns a 200 response when the file has A ) been successfully handled B ) the file it has been sent is empty C ) the file did not decrypt properly . We encountered problems in production with incorrectly encrypted files ( as well as Android generating " rList " files under unknown circumstances ) and the app then uploads them . When the device receives a 200 that is its signal to delete the file . When a file is undecryptable ( this was tracked to a scenario where the device could not create / write an AES encryption key ) we send a 200 response to stop that device attempting to re - upload the data . In the event of a single line being undecryptable ( can happen due to io errors on the device ) we drop only that line ( and store the erroring line in an attempt to track it down . A 400 error means there is something is wrong with the uploaded file or its parameters administrators will be emailed regarding this upload the event will be logged to the apache log . The app should not delete the file it should try to upload it again at some point . If a 500 error occurs that means there is something wrong server side administrators will be emailed and the event will be logged . The app should not delete the file it should try to upload it again at some point . Request format : send an http post request to [ domain name ] / upload remember to include security parameters . Provide the contents of the file encrypted and properly converted to Base64 encoded text as a request parameter entitled " file " . Provide the file name in a request parameter entitled " file_name " .
def upload(OS_API=""):
    file_name = request.values.get("file_name", None)
    if (
            not bool(file_name)
            or file_name.startswith("rList")
            or file_name.startswith("PersistedInstallation")
            or not contains_valid_extension(file_name)
    ):
        return render_template('blank.html'), 200
    s3_file_location = file_name.replace("_", "/")
    participant = get_session_participant()
    if participant.unregistered:
        return render_template('blank.html'), 200
    if FileToProcess.test_file_path_exists(s3_file_location, participant.study.object_id):
        return render_template('blank.html'), 200
    uploaded_file = get_uploaded_file()
    try:
        uploaded_file = decrypt_device_file(uploaded_file, participant)
    except HandledError:
        return render_template('blank.html'), 200
    except DecryptionKeyInvalidError:
        if REPORT_DECRYPTION_KEY_ERRORS:
            tags = {
                "participant": participant.patient_id,
                "operating system": "ios" if "ios" in request.path.lower() else "android",
                "DecryptionKeyError id": str(DecryptionKeyError.objects.last().id),
                "file_name": file_name,
                "bug_report": DECRYPTION_KEY_ADDITIONAL_MESSAGE,
            }
            sentry_client = make_sentry_client(SentryTypes.elastic_beanstalk, tags)
            sentry_client.captureMessage(DECRYPTION_KEY_ERROR_MESSAGE)
        return render_template('blank.html'), 200
    if uploaded_file and file_name and contains_valid_extension(file_name):
        s3_upload(s3_file_location, uploaded_file, participant.study.object_id)
        try:
            FileToProcess.append_file_for_processing(
                s3_file_location, participant.study.object_id, participant=participant
            )
        except ValidationError as e:
            if S3_FILE_PATH_UNIQUE_CONSTRAINT_ERROR in str(e):
                return abort(500)
            else:
                raise
        UploadTracking.objects.create(
            file_path=s3_file_location,
            file_size=len(uploaded_file),
            timestamp=timezone.now(),
            participant=participant,
        )
        return render_template('blank.html'), 200
    elif not uploaded_file:
        return render_template('blank.html'), 200
    else:
        return make_upload_error_report(participant.patient_id, file_name)
def upload ( OS_API = "" ) : file_name = request . values . get ( "file_name" , None ) if ( not bool ( file_name ) or file_name . startswith ( "rList" ) or file_name . startswith ( "PersistedInstallation" ) or not contains_valid_extension ( file_name ) ) : return render_template ( 'blank.html' ) , 200 s3_file_location = file_name . replace ( "_" , "/" ) participant = get_session_participant ( ) if participant . unregistered : return render_template ( 'blank.html' ) , 200 if FileToProcess . test_file_path_exists ( s3_file_location , participant . study . object_id ) : return render_template ( 'blank.html' ) , 200 uploaded_file = get_uploaded_file ( ) try : uploaded_file = decrypt_device_file ( uploaded_file , participant ) except HandledError : return render_template ( 'blank.html' ) , 200 except DecryptionKeyInvalidError : if REPORT_DECRYPTION_KEY_ERRORS : tags = { "participant" : participant . patient_id , "operating system" : "ios" if "ios" in request . path . lower ( ) else "android" , "DecryptionKeyError id" : str ( DecryptionKeyError . objects . last ( ) . id ) , "file_name" : file_name , "bug_report" : DECRYPTION_KEY_ADDITIONAL_MESSAGE , } sentry_client = make_sentry_client ( SentryTypes . elastic_beanstalk , tags ) sentry_client . captureMessage ( DECRYPTION_KEY_ERROR_MESSAGE ) return render_template ( 'blank.html' ) , 200 if uploaded_file and file_name and contains_valid_extension ( file_name ) : s3_upload ( s3_file_location , uploaded_file , participant . study . object_id ) try : FileToProcess . append_file_for_processing ( s3_file_location , participant . study . object_id , participant = participant ) except ValidationError as e : if S3_FILE_PATH_UNIQUE_CONSTRAINT_ERROR in str ( e ) : return abort ( 500 ) else : raise UploadTracking . objects . create ( file_path = s3_file_location , file_size = len ( uploaded_file ) , timestamp = timezone . now ( ) , participant = participant , ) return render_template ( 'blank.html' ) , 200 elif not uploaded_file : return render_template ( 'blank.html' ) , 200 else : return make_upload_error_report ( participant . patient_id , file_name )
Entry point to upload GPS, Accelerometer, Audio, PowerState, Calls Log, Texts Log,
Survey Response, and debugging files to s3.
Entry point to upload GPS Accelerometer Audio PowerState Calls Log Texts Log Survey Response and debugging files to s3 .
""" Entry point to upload GPS, Accelerometer, Audio, PowerState, Calls Log, Texts Log,
    Survey Response, and debugging files to s3.

    Behavior:
    The Beiwe app is supposed to delete the uploaded file if it receives an html 200 response.
    The API returns a 200 response when the file has A) been successfully handled, B) the file it
    has been sent is empty, C) the file did not decrypt properly.  We encountered problems in
    production with incorrectly encrypted files (as well as Android generating "rList" files
    under unknown circumstances) and the app then uploads them.  When the device receives a 200
    that is its signal to delete the file.
    When a file is undecryptable (this was tracked to a scenario where the device could not
    create/write an AES encryption key) we send a 200 response to stop that device attempting to
    re-upload the data.
    In the event of a single line being undecryptable (can happen due to io errors on the device)
    we drop only that line (and store the erroring line in an attempt to track it down.

    A 400 error means there is something is wrong with the uploaded file or its parameters,
    administrators will be emailed regarding this upload, the event will be logged to the apache
    log.  The app should not delete the file, it should try to upload it again at some point.

    If a 500 error occurs that means there is something wrong server side, administrators will be
    emailed and the event will be logged. The app should not delete the file, it should try to
    upload it again at some point.

    Request format:
    send an http post request to [domain name]/upload, remember to include security
    parameters (see user_authentication for documentation). Provide the contents of the file,
    encrypted (see encryption specification) and properly converted to Base64 encoded text,
    as a request parameter entitled "file".
    Provide the file name in a request parameter entitled "file_name". """ # Handle these corner cases first because they requires no database input. # Crash logs are from truly ancient versions of the android codebase # rList are randomly generated by android # PersistedInstallation files come from firebase. # todo: stop uploading junk files in the app by putting our files into a folder. # "Unregistered" participants are blocked from uploading further data. # If the participant is unregistered, throw away the data file, but # return a 200 "OK" status to the phone so the phone decides it can # safely delete the file. # block duplicate FTPs.  Testing the upload history is too complex # when the decryption key is invalid the file is lost.  Nothing we can do. # record the event, send the device a 200 so it can clear out the file. # if uploaded data actually exists, and has a valid extension # race condition: multiple _concurrent_ uploads with same file path. Behavior without # try-except is correct, but we don't care about reporting it. Just send the device a 500 # error so it skips the file, the followup attempt receives 200 code and deletes the file. # Real error is a second validation inside e.error_dict["s3_file_path"]. # Ew; just test for this string instead... # this tells the device to just move on to the next file, try again later. # if the file turns out to be empty, delete it, we simply do not care.
OS_API