Best practice for storing large files (+1GB each) I am looking to build a big data platform for weather forecasting. I will be using a numerical model known as WRF which takes in GFS files and produces files which can be over 1GB each. 

I will be using either AWS or Azure as my cloud provider.

These 1GB files will be processed and a lot of the  variables will be removed, reducing file size, but exactly how much - I do not know. 

Now, the 1GB files may be needed later (it's expensive to run the model), so I do not want to throw them away. 

Do I store them as files on a disk somewhere  (ie the cloud) or do I put them in a DB? Of course, the files will be compressed, but they are still going to be massive...

As for the type of data, the files are binary and the data is like a big spreadsheet but not every row will have the same data... what I mean is that row 1 might have 6 columns whereas row 2 might have 5 columns. To make it worse, the column names are not always identical and some may have a value where others may not. I can however write a script which will ensure the records all have the same column names. I realise some of this information may not be relevant, so I apologise if it is not.
If I understand correctly that you only want to Archive the files produced in case you need the occasional one for referring to later, then cloud file storage would be cheaper and simpler than storing them in a DB.