Running AWS commands works ok, but the code seems messy and I don't like the time.sleep bit
aws_object = json.dumps({"S3Object":{"Bucket":self.s3_bucket,"Name":self.filename}}).replace(' ', '')  # can't have any spaces lol picky AWS CLI
start_textract_command = "aws textract start-document-text-detection --document-location '{}'".format(aws_object)


This is not AWS which is picky, rather you’re not building the command line properly. If you provide a single string and start your subprocess using shell=True, each space will mark a new argument. Instead, you’d be better off building a list of arguments and let subprocess quote them properly:

start_textract_command = ['aws', 'textract', 'start-document-text-detection', '--document-location', json.dumps(…)]


and then just run subprocess.check_output(start_textract_command, stderr=subprocess.STDOUT).

Speaking of which, instead of using the older high level API, you should switch to subprocess.run:

process = subprocess.run(start_textract_command, capture_output=True, encoding='utf-8')
try:
    process.check_returncode()
except CalledProcessError:
    if 'InvalidS3ObjectException' in process.stderr:
       …
else:
    job_id = json.loads(process.stdout)['JobId']
    with open(self.output_filename, 'wb') as output_file:
       process = subprocess.run(['aws', 'textract', 'get-document-text-detection', '--job-id', str(job_id)], stdout=output_file, stderr=subprocess.PIPE)
    if process.returncode:
        log.error(process.stderr.decode('utf-8'))




Now for the time.sleep part, AWS Textract provides two modes of operations: synchronous and asynchronous. You can start by using the synchronous detect-document-text operation if it fits your need and you won't have to deal with the timing at all.

Otherwise, if you need to stick to start-document-text-detection, the completion of the process is published as a notification. I’m not at all versed in how SNS works but they have a tutorial to get you started on using Amazon SNS so you can create a channel, specify it in your textract job and wait for the completion event.