title: How to Create Class Label for Mosaic Augmentation in Image Classification?
body: <p>To create a class label in <code>CutMix</code> or <code>MixUp</code> type augmentation, we can use <code>beta</code> such as <code>np.random.beta</code> or <code>scipy.stats.beta</code> and do as follows for two labels:</p>
  <pre><code>label = label_one*beta + (1-beta)*label_two
  </code></pre>
  <p>But what if we've <strong>more than two</strong> images? In <a href="https://arxiv.org/abs/2004.10934" rel="nofollow noreferrer">YoLo4</a>, they've tried an interesting augmentation called <strong>Mosaic Augmentation</strong> for object detection problems. Unlike <code>CutMix</code> or <code>MixUp</code>, this augmentation creates augmented samples with <strong>4</strong> images. In object detection cases, we can compute the shift of each instance co-ords and thus possible to get the proper ground truth, <a href="https://stackoverflow.com/questions/64335735/how-can-i-get-class-label-from-mosaic-augmentation-in-object-detection-dataloade">here</a>. But for only image classification cases, how can we do that?</p>
  <p>Here is a <strong>starter</strong>.</p>
  <pre><code>import tensorflow as tf
  import matplotlib.pyplot as plt 
  import random
  
  (train_images, train_labels), (test_images, test_labels) = \
  tf.keras.datasets.cifar10.load_data()
  train_images = train_images[:10,:,:]
  train_labels = train_labels[:10]
  train_images.shape, train_labels.shape
  
  ((10, 32, 32, 3), (10, 1))
  </code></pre>
  <p>Here is a function we've written for this augmentation; ( too ugly with an `inner-outer loop! Please suggest if we can do it efficiently.)</p>
  <pre class="lang-py prettyprint-override"><code>def mosaicmix(image, label, DIM, minfrac=0.25, maxfrac=0.75):
      '''image, label: batches of samples 
      '''
      xc, yc  = np.random.randint(DIM * minfrac, DIM * maxfrac, (2,))
      indices = np.random.permutation(int(image.shape[0]))
      mosaic_image = np.zeros((DIM, DIM, 3), dtype=np.float32)
      final_imgs, final_lbs = [], []
  
      # Iterate over the full indices 
      for j in range(len(indices)): 
          # Take 4 sample for to create a mosaic sample randomly 
          rand4indices = [j] + random.sample(list(indices), 3) 
          
          # Make mosaic with 4 samples 
          for i in range(len(rand4indices)):
              if i == 0:    # top left
                  x1a, y1a, x2a, y2a =  0,  0, xc, yc
                  x1b, y1b, x2b, y2b = DIM - xc, DIM - yc, DIM, DIM # from bottom right        
              elif i == 1:  # top right
                  x1a, y1a, x2a, y2a = xc, 0, DIM , yc
                  x1b, y1b, x2b, y2b = 0, DIM - yc, DIM - xc, DIM # from bottom left
              elif i == 2:  # bottom left
                  x1a, y1a, x2a, y2a = 0, yc, xc, DIM
                  x1b, y1b, x2b, y2b = DIM - xc, 0, DIM, DIM-yc   # from top right
              elif i == 3:  # bottom right
                  x1a, y1a, x2a, y2a = xc, yc,  DIM, DIM
                  x1b, y1b, x2b, y2b = 0, 0, DIM-xc, DIM-yc    # from top left
                  
              # Copy-Paste
              mosaic_image[y1a:y2a, x1a:x2a] = image[i,][y1b:y2b, x1b:x2b]
  
          # Append the Mosiac samples
          final_imgs.append(mosaic_image)
          
      return final_imgs, label
  </code></pre>
  <p>The augmented samples, currently with the wrong labels.</p>
  <pre><code>data, label = mosaicmix(train_images, train_labels, 32)
  plt.imshow(data[5]/255)
  </code></pre>
  <p><a href="https://i.stack.imgur.com/JYouf.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/JYouf.png" alt="enter image description here" /></a></p>
  <hr />
  <p>However, here are some more examples to motivate you. Data is from the <a href="https://www.kaggle.com/c/cassava-leaf-disease-classification/overview" rel="nofollow noreferrer">Cassava Leaf</a> competition.</p>
  <p><a href="https://i.stack.imgur.com/f3lSC.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/f3lSC.png" alt="" /></a><br />
  <sub>(source: <a href="https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1984321%2Fb0530886377eb3d7f0c29401ba069ffa%2Fdownload%20(1).png?generation=1607625768667914&amp;alt=media" rel="nofollow noreferrer">googleapis.com</a>)</sub></p>
  <p><a href="https://i.stack.imgur.com/bGJes.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/bGJes.png" alt="" /></a><br />
  <sub>(source: <a href="https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1984321%2Fdfce6cf87b2125c98affa7574ed38b18%2Fdownload%20(3).png?generation=1607625855803714&amp;alt=media" rel="nofollow noreferrer">googleapis.com</a>)</sub></p>