title: Perspective Transform + Crop in iOS with OpenCV
body: <p>I'm trying to implement a cropping &amp;  perspective correction feature into an upcoming app. Whilst doing research, I came across:</p>
  
  <p><a href="https://stackoverflow.com/questions/7838487/executing-cvwarpperspective-for-a-fake-deskewing-on-a-set-of-cvpoint">Executing cv::warpPerspective for a fake deskewing on a set of cv::Point</a></p>
  
  <p><a href="http://sudokugrab.blogspot.ch/2009/07/how-does-it-all-work.html" rel="nofollow noreferrer">http://sudokugrab.blogspot.ch/2009/07/how-does-it-all-work.html</a></p>
  
  <p>So I decided to try implementing this feature with OpenCV - the framework is there so the installation was fast. However, I'm not getting the results I hoped for: (2nd picture is the result)</p>
  
  <p><img src="https://i.stack.imgur.com/EnT2x.png" alt="Original Photo &amp; cropping box"></p>
  
  <p><img src="https://i.stack.imgur.com/qINxd.png" alt="Cropped Photo, bad result"></p>
  
  <p>I've translated all the code to work with Xcode and triple checked the Coordinates. Can you tell me what is wrong with my code? For the sake of completeness, I've also included the UIImage -> Mat conversion + reversal:</p>
  
  <pre><code>- (void)confirmedImage
  {
      if ([_adjustRect frameEdited]) {
  
      cv::Mat src = [self cvMatFromUIImage:_sourceImage];
  
      // My original Coordinates
      // 4-------3
      // |       |
      // |       |
      // |       |
      // 1-------2
  
      CGFloat scaleFactor =  [_sourceImageView contentScale];
      CGPoint p1 = [_adjustRect coordinatesForPoint:4 withScaleFactor:scaleFactor];
      CGPoint p2 = [_adjustRect coordinatesForPoint:3 withScaleFactor:scaleFactor];
      CGPoint p3 = [_adjustRect coordinatesForPoint:1 withScaleFactor:scaleFactor];
      CGPoint p4 = [_adjustRect coordinatesForPoint:2 withScaleFactor:scaleFactor];
  
      std::vector&lt;cv::Point2f&gt; c1;
      c1.push_back(cv::Point2f(p1.x, p1.y));
      c1.push_back(cv::Point2f(p2.x, p2.y));
      c1.push_back(cv::Point2f(p3.x, p3.y));
      c1.push_back(cv::Point2f(p4.x, p4.y));
  
      cv::RotatedRect box = minAreaRect(cv::Mat(c1));
      cv::Point2f pts[4];
      box.points(pts);
  
      cv::Point2f src_vertices[3];
      src_vertices[0] = pts[0];
      src_vertices[1] = pts[1];
      src_vertices[2] = pts[3];
  
      cv::Point2f dst_vertices[4];
      dst_vertices[0].x = 0;
      dst_vertices[0].y = 0;
  
      dst_vertices[1].x = box.boundingRect().width-1;
      dst_vertices[1].y = 0;
  
      dst_vertices[2].x = 0;
      dst_vertices[2].y = box.boundingRect().height-1;
  
      dst_vertices[3].x = box.boundingRect().width-1;
      dst_vertices[3].y = box.boundingRect().height-1;
  
      cv::Mat warpAffineMatrix = getAffineTransform(src_vertices, dst_vertices);
  
      cv::Mat rotated;
      cv::Size size(box.boundingRect().width, box.boundingRect().height);
      warpAffine(src, rotated, warpAffineMatrix, size, cv::INTER_LINEAR, cv::BORDER_CONSTANT);
  
  
      [_sourceImageView setNeedsDisplay];
      [_sourceImageView setImage:[self UIImageFromCVMat:rotated]];
      [_sourceImageView setContentMode:UIViewContentModeScaleAspectFit];
  
      rotated.release();
      src.release();
  
      }
  }
  
  - (UIImage *)UIImageFromCVMat:(cv::Mat)cvMat
  {
      NSData *data = [NSData dataWithBytes:cvMat.data length:cvMat.elemSize()*cvMat.total()];
      CGColorSpaceRef colorSpace;
      if ( cvMat.elemSize() == 1 ) {
          colorSpace = CGColorSpaceCreateDeviceGray();
      }
      else {
          colorSpace = CGColorSpaceCreateDeviceRGB();
      }
      CGDataProviderRef provider = CGDataProviderCreateWithCFData( (__bridge CFDataRef)data );
      CGImageRef imageRef = CGImageCreate( cvMat.cols, cvMat.rows, 8, 8 * cvMat.elemSize(), cvMat.step[0], colorSpace, kCGImageAlphaNone|kCGBitmapByteOrderDefault, provider, NULL, false, kCGRenderingIntentDefault );
      UIImage *finalImage = [UIImage imageWithCGImage:imageRef];
      CGImageRelease( imageRef );
      CGDataProviderRelease( provider );
      CGColorSpaceRelease( colorSpace );
      return finalImage;
  }
  
  - (cv::Mat)cvMatFromUIImage:(UIImage *)image
  {
      CGColorSpaceRef colorSpace = CGImageGetColorSpace( image.CGImage );
      CGFloat cols = image.size.width;
      CGFloat rows = image.size.height;
      cv::Mat cvMat( rows, cols, CV_8UC4 );
      CGContextRef contextRef = CGBitmapContextCreate( cvMat.data, cols, rows, 8, cvMat.step[0], colorSpace, kCGImageAlphaNoneSkipLast | kCGBitmapByteOrderDefault );
      CGContextDrawImage( contextRef, CGRectMake(0, 0, rows, cols), image.CGImage );
      CGContextRelease( contextRef );
      CGColorSpaceRelease( colorSpace );
      return cvMat;
  }
  </code></pre>
  
  <p>Is this the correct approach to my problem? Do you have any sample code that could help me out?</p>
  
  <p>Thank you for reading my question!</p>
  
  <p>UDATE:</p>
  
  <p>I've actually Open Sourced my UIImagePickerController replacement here: <a href="https://github.com/mmackh/MAImagePickerController-of-InstaPDF" rel="nofollow noreferrer">https://github.com/mmackh/MAImagePickerController-of-InstaPDF</a>
  which includes the adjustable cropping view, filters and perspective correction.</p>