title: Camera preview processing on Android
body: <p>I'm making a line follower for my robot on Android (to learn Java/Android programming), currently I'm facing the image processing problem: the camera preview returns an image format called YUV which I want to convert to a threshold in order to know where the line is, how would one do that?</p>
  
  <p>As of now I've succeeded getting <em>something</em>, that is I definitely can read data from the camera preview and by some miracle even know if the light intensity is over or under a certain value at a certain area on the screen. My goal is to draw the robot's path on an overlay over the camera's preview, that too works to some extent, but the problem is the YUV management.</p>
  
  <p><img src="https://i.stack.imgur.com/mrQOg.png" alt="Galaxy Nexus Screenshot"></p>
  
  <p>As you can see not only the dark area is drawn sideways, but it also repeats itself 4 times and the preview image is stretched, I cannot figure out how to fix these problems.</p>
  
  <p>Here's the relevant part of code:</p>
  
  <pre><code>public void surfaceCreated(SurfaceHolder arg0) {
      // TODO Auto-generated method stub
  
      // camera setup
      mCamera = Camera.open();
  
      Camera.Parameters parameters = mCamera.getParameters();
      List&lt;Camera.Size&gt; sizes = parameters.getSupportedPreviewSizes();
      for(int i=0; i&lt;sizes.size(); i++)
      {
          Log.i("CS", i+" - width: "+sizes.get(i).width+" height: "+sizes.get(i).height+" size: "+(sizes.get(i).width*sizes.get(i).height));
      }
  
      // change preview size
      final Camera.Size cs = sizes.get(8);
      parameters.setPreviewSize(cs.width, cs.height);
  
      // initialize image data array
      imgData = new int[cs.width*cs.height];
  
      // make picture gray scale
      parameters.setColorEffect(Camera.Parameters.EFFECT_MONO);
      parameters.setFocusMode(Camera.Parameters.FOCUS_MODE_AUTO);
      mCamera.setParameters(parameters);
  
      // change display size
      LayoutParams params = (LayoutParams) mSurfaceView.getLayoutParams();
      params.height = (int) (mSurfaceView.getWidth()*cs.height/cs.width);
      mSurfaceView.setLayoutParams(params);
  
      LayoutParams overlayParams = (LayoutParams) swOverlay.getLayoutParams();
      overlayParams.width = mSurfaceView.getWidth();
      overlayParams.height = mSurfaceView.getHeight();
      swOverlay.setLayoutParams(overlayParams);
  
      try
      {
          mCamera.setPreviewDisplay(mSurfaceHolder);
          mCamera.setDisplayOrientation(90);
          mCamera.startPreview();
      }
      catch (IOException e)
      {
          e.printStackTrace();
          mCamera.stopPreview();
          mCamera.release();
      }
  
      // callback every time a new frame is available
      mCamera.setPreviewCallback(new PreviewCallback() {
          public void onPreviewFrame(byte[] data, Camera camera)
          {
              // create bitmap from camera preview
              int pixel, pixVal, frameSize = cs.width*cs.height;
              for(int i=0; i&lt;frameSize; i++)
              {
                  pixel = (0xff &amp; ((int) data[i])) - 16;
                  if(pixel &lt; threshold)
                  {
                      pixVal = 0;
                  }
                  else
                  {
                      pixVal = 1;
                  }
                  imgData[i] = pixVal;
              }
  
              int cp = imgData[(int) (cs.width*(0.5+(cs.height/2)))];
  
              //Log.i("CAMERA", "Center pixel RGB: "+cp);
              debug.setText("Center pixel: "+cp);
  
              // process preview image data
              Paint paint = new Paint();
              paint.setColor(Color.YELLOW);
  
  
              int start, finish, last;
              start = finish = last = -1;
              float x_ratio = mSurfaceView.getWidth()/cs.width;
              float y_ratio = mSurfaceView.getHeight()/cs.height;
  
              // display calculated path on overlay using canvas
              Canvas overlayCanvas = overlayHolder.lockCanvas();
              overlayCanvas.drawColor(0, Mode.CLEAR);
  
              // start by finding the tape from bottom of the screen      
              for(int y=cs.height; y&gt;0; y--)
              {
                  for(int x=0; x&lt;cs.width; x++)
                  {
                      pixel = imgData[y*cs.height+x];
  
                      if(pixel == 1 &amp;&amp; last == 0 &amp;&amp; start == -1)
                      {
                          start = x;
                      }
                      else if(pixel == 0 &amp;&amp; last == 1 &amp;&amp; finish == -1)
                      {
                          finish = x;
                          break;
                      }
                      last = pixel;
                  }
                  //overlayCanvas.drawLine(start*x_ratio, y*y_ratio, finish*x_ratio, y*y_ratio, paint);
                  //start = finish = last = -1;
              }
              overlayHolder.unlockCanvasAndPost(overlayCanvas);
          }
      });
  }
  </code></pre>
  
  <p>This code generates an error sometimes when quitting the application due to some method being called after <code>release</code>, which is the least of my problems.</p>
  
  <hr>
  
  <p><strong>UPDATE:</strong></p>
  
  <p>Now that the orientation problem is fixed (CCD sensor orientation) I'm still facing the repetition problem, this is probably related to my YUV data management...</p>
  
  <p><img src="https://i.stack.imgur.com/QEBhF.png" alt="enter image description here"></p>