title: Image processing framework using Android camera2 API and OpenCV
body: <p>I am new to camera2 api. I want to build an image processing framework on my Android phone.</p>
  
  <p><strong>Step1:</strong> use the Camera2 API to open a camera preview stream</p>
  
  <p><strong>Step2:</strong> feed the preview frame data to OpenCV for processing</p>
  
  <p><strong>Step3:</strong> display the processed result live on the screen</p>
  
  <p>Currently, I have finished Step1 using <code>ImageReader</code> and C++ OpenCV code. However, I don't know how to do step3. 
  How do I display the processed image on the screen? (I want to display the normal image, and overlay an icon if I detect predefined object)</p>
  
  <p>Here are some key codes:</p>
  
  <pre><code>protected void createCameraPreview() {
      try {
          SurfaceTexture texture = textureView.getSurfaceTexture();
          assert texture != null;
          texture.setDefaultBufferSize(imageDimension.getWidth(), imageDimension.getHeight());
        //  Surface surface = new Surface(texture);
          Surface mImageSurface = mImageReader.getSurface();
          captureRequestBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
        //  captureRequestBuilder.addTarget(surface);
  
          captureRequestBuilder.addTarget(mImageSurface);
  
  
          cameraDevice.createCaptureSession(Arrays.asList(mImageSurface), new CameraCaptureSession.StateCallback(){
                      @Override
                      public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
                          //The camera is already closed
                          if (null == cameraDevice) {
                              return;
                          }
                          cameraCaptureSessions = cameraCaptureSession;
                          updatePreview();
                      }
                      @Override
                      public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) {
                          Toast.makeText(MainActivity.this, "Configuration change", Toast.LENGTH_SHORT).show();
                      }
                  }, null);
          } catch (CameraAccessException e) {
               e.printStackTrace();
          }
  }
  
  protected void updatePreview() {
      if(null == cameraDevice) {
          Log.e(TAG, "updatePreview error, return");
      }
      try {
          cameraCaptureSessions.setRepeatingRequest(captureRequestBuilder.build(), null, mBackgroundHandler);
      } catch (CameraAccessException e) {
          e.printStackTrace();
      }
  }
  
    private final ImageReader.OnImageAvailableListener mOnImageAvailableListener  = new ImageReader.OnImageAvailableListener() {
  
      @Override
      public void onImageAvailable(ImageReader reader) {
          Log.e(TAG, "onImageAvailable: " + count++);
          Image img = null;
          img = reader.acquireNextImage();
          try {
              if (img == null) throw new NullPointerException("cannot be null");
              ByteBuffer buffer = img.getPlanes()[0].getBuffer();
              byte[] data = new byte[buffer.remaining()];
              buffer.get(data);
              int width = img.getWidth();
              int height = img.getHeight();
  
              // ****try to get captured img for display here (synchronized)
  
              // ****try to process image for detecting the object here (asynchronized)
  
          } catch (NullPointerException ex) {
              ex.printStackTrace();
          }finally {
              Log.e(TAG, "in the finally! ------------");
              if (img != null)
                  img.close();
  
          }
      }
  
  };
  </code></pre>