How to save to disk several 2D np.arrays each having the second dimension variable in size?
You can save these 2D arrays to CSV files.
Concatenating all the arrays into one is up to you: you can concatenate all the arrays into a single one and save only one file, or you can save all the arrays you receive.
I am getting some results in the form of np.arrays of size (no_of_rows, 2), where no_of_rows varies from array to array.
I want to save these results to disk for later usage (plotting). In any way, it doesn't matter how, but if possible in the most efficient way in terms of REOPENING/USING time-consumed-wise.
I can do anything with these 2D arrays. My thought was to create a .npy file containing a 3D tensor of shape (no_of_matrices, ? , 2), but that ? is my problem. Each 2D array can potentially have a different no_of_rows. So I abandoned this idea.
I now made a list of these 2D np.arrays. I now want to save that list to disk.
I have read about np.savez, but this doesn't preserve the order of saving. That is, when loading back in memory, I won't know which array is which, apart from many if's statements to check the no_of_lines of each array and match it to what it means for when plotting it later.
Do I have any chance to store these arrays in 1 SINGLE file, or do I just have to resort to creating multiple files, named in a distinctive way, one for each 2D array, and then access these files individually when plotting?
[PS: I can only have a maximum of 100-ish of such 2D arrays, usually having less than 10.]
Thanks