title: How to load an image dataset in python with a manually defined MasterImage class?
body: <p>I have a folder, Dataset, with the Train and Test subfolders containing images.
  for example, in the Train folder, I have three subfolders, Cat, Car and Man each containing about 1700 images.</p>
  <p>so I am using this class definition to load and render the images in the format suitable for a deep learning model.
  It gives me a wrong shape -  shape=(0, 200, 200, 3), dtype=float64) for my train images while I expect it to be (shape=(1700, 200, 200, 3), dtype=float64)</p>
  <p>My code</p>
  <pre><code>import os
  import pickle
  import numpy as np
  
  path_train = &quot;C:\\Users\\me\\Jupiter_Notebooks\\Dataset\\Train&quot;
  path_test = &quot;C:\\Users\\me\\Jupiter_Notebooks\\Dataset\\Test&quot;
  
  # defining the class
  
  class MasterImage(object):
  
      def __init__(self,PATH='', IMAGE_SIZE = 100):
          self.PATH = PATH
          self.IMAGE_SIZE = IMAGE_SIZE
  
          self.image_data = []
          self.x_data = []
          self.y_data = []
          self.CATEGORIES = []
  
          # This will get List of categories
          self.list_categories = []
  
      def get_categories(self):
          for path in os.listdir(self.PATH):
              if '.DS_Store' in path:
                  pass
              else:
                  self.list_categories.append(path)
          print(&quot;Found Categories &quot;,self.list_categories,'\n')
          return self.list_categories
  
      def Process_Image(self):
          try:
              &quot;&quot;&quot;
              Return Numpy array of image
              :return: X_Data, Y_Data
              &quot;&quot;&quot;
              self.CATEGORIES = self.get_categories()
              for categories in self.CATEGORIES:                                                  # Iterate over categories
  
                  train_folder_path = os.path.join(self.PATH, categories)                         # Folder Path
                  class_index = self.CATEGORIES.index(categories)                                 # this will get index for classification
  
                  for img in os.listdir(train_folder_path):                                       # This will iterate in the Folder
                      new_path = os.path.join(train_folder_path, img)                             # image Path
  
                      try:        # if any image is corrupted
                          image_data_temp = cv2.imread(new_path)                 # Read Image as numbers
                          image_temp_resize = cv2.resize(image_data_temp,(self.IMAGE_SIZE,self.IMAGE_SIZE))
                          self.image_data.append([image_temp_resize,class_index])
                          random.shuffle(self.image_data)
                      except:
                          pass
  
              data = np.asanyarray(self.image_data)
  
              # Iterate over the Data
              for x in data:
                  self.x_data.append(x[0])        # Get the X_Data
                  self.y_data.append(x[1])        # get the label
  
              X_Data = np.asarray(self.x_data) / (255.0)      # Normalize Data
              Y_Data = np.asarray(self.y_data)
  
              # reshape x_Data
  
              X_Data = X_Data.reshape(-1, self.IMAGE_SIZE, self.IMAGE_SIZE, 3)
  
              return X_Data, Y_Data
          except:
              print(&quot;Failed to run Function Process Image &quot;)
  
      def pickle_image(self):
  
          &quot;&quot;&quot;
          :return: None Creates a Pickle Object of DataSet
          &quot;&quot;&quot;
          # Call the Function and Get the Data
          X_Data,Y_Data = self.Process_Image()
  
          # Write the Entire Data into a Pickle File
          pickle_out = open('X_Data','wb')
          pickle.dump(X_Data, pickle_out)
          pickle_out.close()
  
          # Write the Y Label Data
          pickle_out = open('Y_Data', 'wb')
          pickle.dump(Y_Data, pickle_out)
          pickle_out.close()
  
          print(&quot;Pickled Image Successfully &quot;)
          return X_Data,Y_Data
  
      def load_dataset(self):
  
          try:
              # Read the Data from Pickle Object
              X_Temp = open('X_Data','rb')
              X_Data = pickle.load(X_Temp)
  
              Y_Temp = open('Y_Data','rb')
              Y_Data = pickle.load(Y_Temp)
  
              print('Reading Dataset from Pickle Object')
  
              return X_Data,Y_Data
  
          except:
              print('Could not Found Pickle File ')
              print('Loading File and Dataset  ..........')
  
              X_Data,Y_Data = self.pickle_image()
              return X_Data,Y_Data
  
  
  # loading data
  
  dstrain = MasterImage(PATH = path_train,IMAGE_SIZE = 100)
  dstrain
  
  (train_images, train_labels) = dstrain.load_dataset()
  
  
  print('Train: X_images=%s, y_labels=%s' % (train_images.shape, train_labels.shape))
  Train: X_images=(0, 100, 100, 3), y_labels=(0,)
  
  </code></pre>
  <p>what can I change to get the correct shape? thank you very much.</p>
  <p>my data is similar to the training set from here.</p>
  <p><a href="https://github.com/soumilshah1995/Smart-Library-to-load-image-Dataset-for-Convolution-Neural-Network-Tensorflow-Keras-/tree/master/training_set" rel="nofollow noreferrer">https://github.com/soumilshah1995/Smart-Library-to-load-image-Dataset-for-Convolution-Neural-Network-Tensorflow-Keras-/tree/master/training_set</a></p>