Which one is faster in production? file on disk vs file in memory(StringIO,BytesIO)
Writing and reading from memory is fast. But keep in mind that you have tons of data. So storing all that in-memory might take up all your memory and might make the system slow or might throw errors due to Out of Memory. So, analyze and understand which all data to be put in memory and which all to be written to files.
I am converting a dictionary to pandas object with to_csv. I have both way of doing this 
1 - by writing file in disk(with open statement)
2 - by writing in memory (StringIO,BytesIO) 
I have used it in both way creating file in disk and using StringIO to convert to pandas object. I tried to read comparisons between these three, but bit confused which one is faster so i can use it in production to process tons of data.