title: opencv feature matching with empty formular template
body: <p>i've been trying to match a scanned formular with its empty template. The goal is to rotate and scale it to match the template.  </p>
  
  <p><a href="https://i.stack.imgur.com/7xTa9.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7xTa9.jpg?s=128" alt="Source fomular, to be rescaled and rotated"></a>
  <a href="https://i.stack.imgur.com/TAYUw.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/TAYUw.jpg?s=128" alt="Template"></a><br>
  <code>Source (left), template (right)</code><br>
  <a href="https://i.stack.imgur.com/UDWr3.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/UDWr3.jpg?s=128" alt="Matches"></a>
  <a href="https://i.stack.imgur.com/s1QTR.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/s1QTR.jpg?s=128" alt="Homography warp result"></a><br>
  <code>Match (left), Homography warp (right)</code><br>
  The <strong>template does not contain any very specific logo, fixation cross or rectangular frame</strong> that would conveniently help me with feature or pattern matching. Even worse, the scanned formular can be skewed, altered and contains handwritten signatures and stamps.</p>
  
  <p>My approach, after unsuccessfully testing ORB feature matching, was to concentrate on the shape of the formular (lines and column).</p>
  
  <p>The pictures I provide here are obtained by reconstituting lines after a segment detection (LSD) with a certain minimum size. Most of what remains for source and template is the document layout itself.</p>
  
  <p>In the following script (that should work out of the box along with pictures), I attempt to do ORB feature matching, but fail to make it work because it is concentrating on edges and not on the document layout.</p>
  
  <pre><code>import cv2  # using opencv-python v3.4
  import numpy as np
  from imutils import resize
  
  
  # alining image using ORB descriptors, then homography warp
  def align_images(im1, im2,MAX_MATCHES=5000,GOOD_MATCH_PERCENT = 0.15):
  
      # Detect ORB features and compute descriptors.
      orb = cv2.ORB_create(MAX_MATCHES)
      keypoints1, descriptors1 = orb.detectAndCompute(im1, None)
      keypoints2, descriptors2 = orb.detectAndCompute(im2, None)
  
      # Match features.
      matcher = cv2.DescriptorMatcher_create(cv2.DESCRIPTOR_MATCHER_BRUTEFORCE_HAMMING)
      matches = matcher.match(descriptors1, descriptors2, None)
  
      # Sort matches by score
      matches.sort(key=lambda x: x.distance, reverse=False)
  
      # Remove not so good matches
      numGoodMatches = int(len(matches) * GOOD_MATCH_PERCENT)
      matches = matches[:numGoodMatches]
  
      # Draw top matches
      imMatches = cv2.drawMatches(im1, keypoints1, im2, keypoints2, matches, None)
  
      # Extract location of good matches
      points1 = np.zeros((len(matches), 2), dtype=np.float32)
      points2 = np.zeros((len(matches), 2), dtype=np.float32)
  
      for i, match in enumerate(matches):
          points1[i, :] = keypoints1[match.queryIdx].pt
          points2[i, :] = keypoints2[match.trainIdx].pt
  
      # Find homography
      h, mask = cv2.findHomography(points1, points2, cv2.RANSAC)
  
      # Use homography
      if len(im2.shape) == 2:
          height, width = im2.shape
      else:
          height, width, channels = im2.shape
      im1Reg = cv2.warpPerspective(im1, h, (width, height))
  
      return im1Reg, h, imMatches
  
  template_fn = './stack/template.jpg'
  image_fn = './stack/image.jpg'
  
  im = cv2.imread(image_fn, cv2.IMREAD_GRAYSCALE)
  template = cv2.imread(template_fn, cv2.IMREAD_GRAYSCALE)
  
  # aligh images
  imReg, h, matches = align_images(template,im)
  
  # display output
  cv2.imshow('im',im)
  cv2.imshow('template',template)
  cv2.imshow('matches',matches)
  cv2.imshow('result',imReg)
  cv2.waitKey(0)
  cv2.destroyAllWindows()
  </code></pre>
  
  <p>Is there any way to make the pattern matching algorithm work on the image on the left (source)? (another idea was to leave only lines intersections)</p>
  
  <p>Alternatively, I have been trying to do scale and rotation invariant pattern matching for loops and while keeping max correlation, but it is way too resource consuming and not very reliable.</p>
  
  <p>I'm therefore looking for hints in the right direction using opencv.</p>