memory error while converting images into an array
You can actually compute how much memory it will take to hold 57,000 images in memory (it is a lot!). You are also holding them twice: once in the features list and then trying again in npfeatures. This second part will make a full copy of the entire features list. Hence why it runs out of memory there.

Here are some starting steps that should help you understand the limitations of your approach and perhaps get a working method:

1. You don't need this line: x = np.expand_dims(x, axis=0):

They do that in the Keras documentation, because you need a single image to have an additional dimension for the batch size. You do not need that because you append many images to a list (features), and the length of that list is the same thing, the batch size (the number of images).

2. Do your own scaling

In the case f VGG16 (and 19 I believe), the preprocess_input method simply scales pixel values between -1 and +1. You can probably do this a little more light-weight yourself. You can do the scaling on a numpy array like this:

x = (x / 127.5 - 1)


... and now remove the line with preprocess_input

3. Break it down into smaller pieces:

You can then either feed them directly into you models (if that is possible), or save the npfeatures to disk, one chunk at a time.

4. Try seeing how much memory your machine has while running this script:

Add a simple line after features.append(x), like this:

print("Loaded {} images"format(len(features)))


If you are running this on a Linux machine or Mac OSX, try using a tool like htop in terminal. There is also system monitor... same for Windows. You should be able to see the memory consumption grow until the point your script crashes. Now you know how many images you can do in one cycle - the last printed number before the crash.