How many images can be trained in Google Colab? I am using the ResNet50 pretrained model to train my images using TensorFlow. I have 70k images and upgraded to Google Colab Pro, but still I am facing a memory error. So how many images I can train in Google Colab? And how much RAM is needed for 70k images?
This is how I labeled and loaded images from the drive.
labels = []

imagePaths_generater = paths.list_images(Config.DATASET_PATH)
imagePaths = []
for item in imagePaths_generater:
  imagePaths.append(item)

for imagePath in imagePaths:
  label = imagePath.split(os.path.sep)[-2].split(&quot;_&quot;)
  #print(label)
  image = load_img(imagePath, target_size=(224, 224))
  image = img_to_array(image)
  image = preprocess_input(image)
  data.append(image)
  labels.append(label)
 

data = np.array(data, dtype=&quot;float&quot;)
labels = np.array(labels)```
The real problem is that you should not try to fit all your images in memory. Instead, you should small groups of images, normally called &quot;minibatches&quot;, which can fit in the GPU/CPU memory.
For that, tensorflow offers the function tf.keras.preprocessing.image_dataset_from_directory that loads images from a directory. I suggest you take a look at Tensorflow's tutorial for image loading, which guides you to the image loading process, as well as the training of a model with those images.