title: Detect rectangular portraits of people on images with OpenCV
body: <p>I have many images of yearbooks with people portraits and I'm trying to build an algorytm that will detect those portraits. At least, to detect correct rectangular portraits. <a href="https://i.stack.imgur.com/twHVm.jpg" rel="noreferrer">Example 1</a> <a href="https://i.stack.imgur.com/08KXn.jpg" rel="noreferrer">Example 2</a></p>
  
  <p>I'm trying to investigate three directions:</p>
  
  <ol>
  <li>Face detection</li>
  <li>Dark rectangles detection (Since portraits are usually darker shapes on brighter background)</li>
  <li>People name extraction from OCR'ed texts</li>
  </ol>
  
  <p>By combining results of three algorithms above, I hope to get some methodology, that will be applicable for many different yearbooks pages.</p>
  
  <p>I would be very appreciate for any help for the rectangles detection.
  I started with Java and OpenCV 3.</p>
  
  <p>Here is my code applied for <a href="https://i.stack.imgur.com/twHVm.jpg" rel="noreferrer">an image</a>:</p>
  
  <pre class="lang-java prettyprint-override"><code>System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
  Mat source = Imgcodecs.imread("Path/to/image", Imgcodecs.CV_LOAD_IMAGE_ANYCOLOR);
  Mat destination = new Mat(source.rows(), source.cols(), source.type());
  
  Imgproc.cvtColor(source, destination, Imgproc.COLOR_RGB2GRAY);
  Imgproc.GaussianBlur(destination, destination, new Size(5, 5), 0, 0, Core.BORDER_DEFAULT);
  
  int threshold = 100;
  Imgproc.Canny(destination, destination, 50, 100);
  Imgproc.Canny(destination, destination, threshold, threshold*3);
  </code></pre>
  
  <p>At this point, I have such result: 
  <a href="https://i.stack.imgur.com/KoxG5.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/KoxG5.jpg" alt="enter image description here"></a></p>
  
  <p>Trying to find contours from the edges above:</p>
  
  <pre class="lang-java prettyprint-override"><code>    List&lt;MatOfPoint&gt; contourDetections = new ArrayList&lt;&gt;();
      Mat hierarchy = new Mat();
  
      // Find contours
      Imgproc.findContours(destination, contourDetections, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
  
      // Draw contours 
      Imgproc.drawContours(source, contours, -1, new Scalar(255,0,0), 2);
  </code></pre>
  
  <p>Getting this result:
  <a href="https://i.stack.imgur.com/nDRMp.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/nDRMp.jpg" alt="enter image description here"></a></p>
  
  <p>But not sure how to extract rectangles from those contours since many of lines are incomplete. </p>
  
  <p>Getting back to edges and trying to find vertical and horizontal lines using HoughLinesP:</p>
  
  <pre class="lang-java prettyprint-override"><code>    Mat lines = new Mat();
      int thre = 50;
      int minLineSize = 250;
      int lineGap = 80;
  
      int ignoreLinesShorter = 300;
  
      Imgproc.HoughLinesP(destination, lines, 1, Math.PI/180, thre, minLineSize, lineGap);
  
      for(int c = 0; c &lt; lines.rows(); c++) {
  
          double[] vec = lines.get(c, 0);
  
          double  x1 = vec[0],
                  y1 = vec[1],
                  x2 = vec[2],
                  y2 = vec[3];
  
          // Filtering only verticat and horizontal lines
          if(x1 == x2 || y1 == y2) {
  
              // Filtering out short lines
              if(Math.abs(x1 - x2) &gt; ignoreLinesShorter || Math.abs(y1 - y2) &gt; ignoreLinesShorter) {
  
                Point start = new Point(x1, y1);
                Point end = new Point(x2, y2);
  
                // Draw line
                Imgproc.line(source, start, end, new Scalar(0,0,255), 2);
              }
          }
      }
  </code></pre>
  
  <p>Result:</p>
  
  <p><a href="https://i.stack.imgur.com/CdADt.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/CdADt.jpg" alt="enter image description here"></a></p>
  
  <p>Like with contours, I'm still not seeing correct rectangles that I could detect. Could you help me with a correct direction? Maybe there is an easier way to perform this task?</p>