How to deal with memory exceptions while processing large amount of data?
I had also faced similar issues when my python script was processing my 12GB and 35GB files. It used take weeks for processing and many times failed.  

You need to check your mathematical operation, if it can work on subset of data. In my case, I finally was able to split the file in multiple pieces and run the mathematical operation on individual pieces. It helped improving the speed as well as better exceptional handling.  

If you cant split your file, then check if your code can work in batches. Then you can read your file in chunks (in pd.read_csv you can supply chunk size) and then feed to your formula piece by piece.