Pandas memory error when saving DataFrame to file
There are several options. You can pickle the dataframe or you can use hdf5 format. 
These will occupy less memory. Also when you load it next time, it would be quicker then other formats.
I finnally managed to join two big DataFrames on a big machine of my school (512G memory). At the moment we re two people using the same machine, the other one is using about 120G of the memory, after I called the garbage collecter we get to 420G.
I want to save the DataFrame to memory so I then I can reuse it easily and move it on another machine, I have tried to export it to a parquet file, but I get a memory error...
So how can I manage to dump that Dataframe on the hard drive in purpose of reusing it without running into memory error when memory is already near full ?
Thank you