title: Semantic Image Segmentation with colored masks
body: <p>So i have a set of pictures with their masks in color for example color blue is for chair, red for lamps, etc.</p>
  
  <p>As I am new to all of this i have tried doing doing this with the unet model, i have processed the images with keras and like this.</p>
  
  <pre><code>def data_generator(img_path,mask_path,batch_size):
      c=0
      n = os.listdir(img_path)
      m = os.listdir(mask_path)
      random.shuffle(n)
      while(True):
          img = np.zeros((batch_size,256,256,3)).astype("float")
          mask = np.zeros((batch_size,256,256,1)).astype("float")
  
          for i in range(c,c+batch_size):
              train_img = cv2.imread(img_path+"/"+n[i])/255.
              train_img = cv2.resize(train_img,(256,256))
              img[i-c] = train_img
  
              train_mask = cv2.imread(mask_path+"/"+m[i],cv2.IMREAD_GRAYSCALE)/255.
              train_mask = cv2.resize(train_mask,(256,256))
              train_mask = train_mask.reshape(256,256,1)
  
              mask[i-c]=train_mask
  
          c+=batch_size
          if(c+batch_size&gt;=len(os.listdir(img_path))):
              c=0
              random.shuffle(n)
  
          yield img,mask
  </code></pre>
  
  <p>Now looking closer I think this way wont work with my masks, i tried processing the masks as rgb color but my model wont train like that.</p>
  
  <p>model.</p>
  
  <pre><code>def unet(pretrained_weights = None,input_size = (256,256,3)):
      inputs = Input(input_size)
      conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)
      conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1)
      pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
      conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1)
      conv2 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2)
      pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
      conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool2)
      conv3 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv3)
      pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)
      conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool3)
      conv4 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv4)
      drop4 = Dropout(0.5)(conv4)
      pool4 = MaxPooling2D(pool_size=(2, 2))(drop4)
  
      conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4)
      conv5 = Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5)
      drop5 = Dropout(0.5)(conv5)
  
      up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))
      merge6 = concatenate([drop4,up6], axis = 3)
      conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6)
      conv6 = Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6)
  
      up7 = Conv2D(256, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv6))
      merge7 = concatenate([conv3,up7], axis = 3)
      conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7)
      conv7 = Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7)
  
      up8 = Conv2D(128, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv7))
      merge8 = concatenate([conv2,up8], axis = 3)
      conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8)
      conv8 = Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8)
  
      up9 = Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv8))
      merge9 = concatenate([conv1,up9], axis = 3)
      conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9)
      conv9 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)
      conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)
      conv10 = Conv2D(1, 1, activation = 'sigmoid')(conv9)
  
      model = Model(input = inputs, output = conv10)
  
      model.compile(optimizer = Adam(lr = 1e-4), loss = 'binary_crossentropy', metrics = ['accuracy'])
  
      #model.summary()
  
      if(pretrained_weights):
          model.load_weights(pretrained_weights)
  
      return model
  </code></pre>
  
  <p>So my question is how do I train a model with colored image masks.</p>
  
  <p>Edit, example of the data i have.</p>
  
  <p>given image to train the model
  <a href="https://i.stack.imgur.com/qeR2b.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/qeR2b.jpg" alt="example of given image to train model"></a></p>
  
  <p>mask of it
  <a href="https://i.stack.imgur.com/rrL18.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/rrL18.png" alt="mask of image"></a></p>
  
  <p>and the percentage of every mask like this.
  <code>{"water": 4.2, "building": 33.5, "road": 0.0}</code></p>