Using Numpy.Load - is this the fastest method? It seems slow
As an approximate test,
I can load a .npy file with 1_000_000 elements (int64) in 6 milliseconds (from my local disk).

Are your files (/content/drive/...) in a Google data center, or on your own storage?
In the latter case, can you copy to files, so they're at Google, along with the compute?
I am loading 10-15MB numpy arrays saved in .npy format in a loop, and each load takes about 1.5s in Google Colab. Is that an acceptable speed? Its by far the slowest part of my loop. Also, the second time I run the loop its faster (presumably because the loaded files were saved to RAM?) but then eventually it slows back down again (runs out of RAM?).
Here is the code I am using to load the arrays:
data = np.load('/content/drive/My Drive/Share/Daily Data/Processed/' + str(fileName))
Any advice to speed this up would be greatly appreciated, right now I have load 1500 arrays per loop so each loop takes around 25 minutes. The arrays must be loaded separately (please don't suggest I append them all and load it all at once, I can't).