Track the existence of process I have an api endpoint that, when called, initiates the generation of a report. This is time consuming process and response could come pretty late. So, we came up with caching architecture and once finished the response is saved to redis.
However, on the frontend side we dicided to make a request once in a while, until it is ready. To face the issue of overlapping queries when one report is initialized several times in a row, I decided to make a temporary file, that exists only when the report is being generated, and the api response is &quot;This report is already being generated&quot;. Once the generation is finished, file is deleted and response is already cached.
This is mockup of what I am doing
def post(self):
   serialized_data = self.serializer().deserialize({
            'task': json.loads(request.values.get('task')),
            'media_company': json.loads(request.values.get('mediaCompany'))
    })

   if not os.path.exists('/root/progress_data'):
        os.mkdir('/root/progress_data')

    self.build_range(serialized_data)

    serialized_data['media_company']['date_from'] = \
                                        self.date_from.strftime(&quot;%Y-%m-%d&quot;)
    serialized_data['media_company']['date_to'] = \
                                        self.date_to.strftime(&quot;%Y-%m-%d&quot;)
    progress_hash = hashlib.sha256(str(serialized_data).encode()).hexdigest()

    if self.progress_check(progress_hash):
        return json_response({
            &quot;success&quot;: False,
            'result': &quot;This report is already being generated&quot;,
        })

    file = open(f'/root/progress_data/{progress_hash}', 'w+')
    file.close()
    try:
        report = self.generate_report(serialized_data)
    except:
        os.remove(f'/root/progress_data/{progress_hash}')
        return json_response({
            &quot;success&quot;: False,
            'result': &quot;Error while generating report&quot;,
        })
    os.remove(f'/root/progress_data/{progress_hash}')
    return json_response({
        &quot;success&quot;: True,
        'data': report,
    })

I think that this is not really production-ready solution, but I could not come up with something better.
Also, there are some holes where the file is not being deleted in all cases.
Could you show me potential holes and may be another way of checking the progress
There is nothing I can test here, so I will only make a couple superficial remarks.
You do not offer a lot of insight into your application. If you say you have a problem with overlapping queries, then you should address the root of the problem. I have the impression that you simply devised a workaround. It may do the job but at the expense of performance or reliability. It could also create more problems.
Repetition
What is immediately apparent is that you have a hardcoded path repeated multiple times across your code.
At the top of your code you should define a variable for '/root/progress_data'. Then if you decide to change the file name or path you will have only one line to change.
But Python has a library for temp files: https://docs.python.org/3/library/tempfile.html. This is what you should be using.
The date format &quot;%Y-%m-%d&quot; could be made a variable too.
Security
Does your program really need to run as root ?
Maximize your try/catch/finally blocks
You repeat this twice:
os.remove(f'/root/progress_data/{progress_hash}')

To ensure that the file gets removed at the end of the process you could place this line of code in a finally block.
Same goes for the return value:
return json_response({
    &quot;success&quot;: False,
    'result': &quot;Error while generating report&quot;,
})

return json_response({
    &quot;success&quot;: True,
    'data': report,
})

Keep only one return but use variables for the response depending on the outcome of the execution.
Thus your code will be shorter, more straightforward and easier to comprehend. But I still think the idea is wrong.
I would probably use a database instead of temp files. It can be a small self-contained DB file like SQLite. You need a simple table with details of the job, some kind of unique identifier and a column to update the status of the job. Then your API can query the table and say &quot;ready&quot; when the column has been to updated to the finished status.
This looks clumsy to me. I would ditch the idea and go back to the drawing board. If you are stuck my suggestion would be to post on Stack Overflow for guidance but add more context to your question, explain how the API is used, what your constraints are.