out of memory error when consrtucting 2d list from 2 numpy arrays
I would recommend breaking the problem down a little bit to reduce the memory usage at any one time.

The first part of your main function gets the IDs using getIDList. That seems fine, so leave it there.

I would then break that list down into smaller chunks, calling get3Dmatrix on each chunk in turn. Altering your code, it might look something like this:

# Get number of entries in ID list
N = len(ID_list)

# break it down into a number of chunks e.g. 4, based on your progress bar
import numpy as np    # should already be imported

N = len(ID_list)
num_chunks = 4           # you can play with this number, making it larger until you don't get emmory errors
chunks = np.linspace(0, N, num_chunks)

for i in range(len(chunks) - 1):
    this_sublist = ID_list[chunks[i] : chunks[i + 1]]
    sub_data_set = get3Dmatrix(this_sublist)

    # At this point, either save this sub_data_set, or try appending it to another list toi make one final numpy matrix at the end before saving

...

print("Begin saving in numpy file")
np.save(output_path+'np_ds(10000)-25-25-25(zyx)_one_hot.npy', Data_set)
print("%s time takes in seconds" % (time.time() - start_time))


Even from the traceback you added, it is hard to say where exactly in your code that is happening.

Roughly looking at the dimensions you mention, it also doesn't seem plausible the a 16Gb machine is running out of memory - so I must not completely understand just how many images/patches are being saved.