title: Scaling YUV420 Image using libyuv produces weird output
body: <p>Possible duplicate of <a href="https://stackoverflow.com/questions/42960251/problems-when-scaling-a-yuv-image-using-libyuv-library">This</a> question with major parts picked from <a href="https://stackoverflow.com/questions/19677747/how-to-find-out-resolution-and-count-of-frames-in-yuv-420-file">here</a>. I've tried whatever solutions were provided there, they don't work for me.</p>
  
  <p><strong>Background</strong></p>
  
  <p>I'm capturing an image in <a href="https://developer.android.com/reference/android/graphics/ImageFormat.html#YUV_420_888" rel="nofollow noreferrer">YUV_420_888</a> image format returned from ARCore's <code>frame.acquireCameraImage()</code> method. Since I've set the camera configuration at 1920*1080 resolution, I need to scale it down to 224*224 to pass it to my tensorflow-lite implementation. I do that by using <a href="https://chromium.googlesource.com/libyuv/libyuv/" rel="nofollow noreferrer">LibYuv</a> library through the Android NDK.</p>
  
  <p><strong>Implementation</strong></p>
  
  <p>Prepare the image frames</p>
  
  <pre><code>    //Figure out the source image dimensions
      int y_size = srcWidth * srcHeight;
  
      //Get dimensions of the desired output image
      int out_size = destWidth * destHeight;
  
      //Generate input frame
      i420_input_frame.width = srcWidth;
      i420_input_frame.height = srcHeight;
      i420_input_frame.data = (uint8_t*) yuvArray;
      i420_input_frame.y = i420_input_frame.data;
      i420_input_frame.u = i420_input_frame.y + y_size;
      i420_input_frame.v = i420_input_frame.u + (y_size / 4);
  
      //Generate output frame
      free(i420_output_frame.data);
      i420_output_frame.width = destWidth;
      i420_output_frame.height = destHeight;
      i420_output_frame.data = new unsigned char[out_size * 3 / 2];
      i420_output_frame.y = i420_output_frame.data;
      i420_output_frame.u = i420_output_frame.y + out_size;
      i420_output_frame.v = i420_output_frame.u + (out_size / 4);
  </code></pre>
  
  <p>I scale my image using Libyuv's <code>I420Scale</code> method</p>
  
  <pre><code>libyuv::FilterMode mode = libyuv::FilterModeEnum::kFilterBox;
  jint result = libyuv::I420Scale(i420_input_frame.y, i420_input_frame.width,
                                  i420_input_frame.u, i420_input_frame.width / 2,
                                  i420_input_frame.v, i420_input_frame.width / 2,
                                  i420_input_frame.width, i420_input_frame.height,
                                  i420_output_frame.y, i420_output_frame.width,
                                  i420_output_frame.u, i420_output_frame.width / 2,
                                  i420_output_frame.v, i420_output_frame.width / 2,
                                  i420_output_frame.width, i420_output_frame.height,
                                  mode);
  </code></pre>
  
  <p>and return it to java</p>
  
  <pre><code>    //Create a new byte array to return to the caller in Java
      jbyteArray outputArray = env -&gt; NewByteArray(out_size * 3 / 2);
      env -&gt; SetByteArrayRegion(outputArray, 0, out_size, (jbyte*) i420_output_frame.y);
      env -&gt; SetByteArrayRegion(outputArray, out_size, out_size / 4, (jbyte*) i420_output_frame.u);
      env -&gt; SetByteArrayRegion(outputArray, out_size + (out_size / 4), out_size / 4, (jbyte*) i420_output_frame.v);
  </code></pre>
  
  <p>Actual image : <a href="https://i.stack.imgur.com/uDN7h.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/uDN7h.jpg" alt="enter image description here"></a></p>
  
  <p>What it looks like post scaling :</p>
  
  <p><a href="https://i.stack.imgur.com/ivxyq.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ivxyq.jpg" alt="enter image description here"></a></p>
  
  <p>What it looks like if I create an Image from the <code>i420_input_frame</code> without scaling :</p>
  
  <p><a href="https://i.stack.imgur.com/MSRJb.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/MSRJb.jpg" alt="enter image description here"></a></p>
  
  <p>Since the scaling messes up the colors big time, tensorflow fails to recognize objects properly. (It recognizes properly in their sample application) <strong>What am I doing wrong to mess up the colors big time?</strong></p>