title: Using opencv to find the most similar image that contains another image
body: <p>If the title isn't clear let's say I have a list of images (10k+), and I have a target image I am searching for.</p>
  <p>Here's an example of the target image:</p>
  <p><a href="https://i.stack.imgur.com/5HLSN.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/5HLSN.png" alt="enter image description here" /></a></p>
  <p>Here's an example of images I will want to be searching to find something 'similar' (ex1, ex2, and ex3):</p>
  <p><a href="https://i.stack.imgur.com/kQrgS.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/kQrgS.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.stack.imgur.com/7OyyW.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7OyyW.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.stack.imgur.com/znD6A.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/znD6A.png" alt="enter image description here" /></a></p>
  <p>Here's the matching I do (I use KAZE)</p>
  <pre><code>from matplotlib import pyplot as plt
  import numpy as np
  import cv2
  from typing import List
  import os
  import imutils
  
  
  def calculate_matches(des1: List[cv2.KeyPoint], des2: List[cv2.KeyPoint]):
      &quot;&quot;&quot;
      does a matching algorithm to match if keypoints 1 and 2 are similar
      @param des1: a numpy array of floats that are the descriptors of the keypoints
      @param des2: a numpy array of floats that are the descriptors of the keypoints
      @return:
      &quot;&quot;&quot;
      # bf matcher with default params
      bf = cv2.BFMatcher(cv2.NORM_L2)
      matches = bf.knnMatch(des1, des2, k=2)
      topResults = []
      for m, n in matches:
          if m.distance &lt; 0.7 * n.distance:
              topResults.append([m])
  
      return topResults
  
  
  def compare_images_kaze():
      cwd = os.getcwd()
      target = os.path.join(cwd, 'opencv_target', 'target.png')
      images_list = os.listdir('opencv_images')
      for image in images_list:
          # get my 2 images
          img2 = cv2.imread(target)
          img1 = cv2.imread(os.path.join(cwd, 'opencv_images', image))
          for i in range(0, 360, int(360 / 8)):
              # rotate my image by i
              img_target_rotation = imutils.rotate_bound(img2, i)
  
              # Initiate KAZE object with default values
              kaze = cv2.KAZE_create()
              kp1, des1 = kaze.detectAndCompute(img1, None)
              kp2, des2 = kaze.detectAndCompute(img2, None)
              matches = calculate_matches(des1, des2)
  
              try:
                  score = 100 * (len(matches) / min(len(kp1), len(kp2)))
              except ZeroDivisionError:
                  score = 0
              print(image, score)
              img3 = cv2.drawMatchesKnn(img1, kp1, img_target_rotation, kp2, matches,
                                        None, flags=2)
              img3 = cv2.cvtColor(img3, cv2.COLOR_BGR2RGB)
              plt.imshow(img3)
              plt.show()
              plt.clf()
  
  
  if __name__ == '__main__':
      compare_images_kaze()
  </code></pre>
  <p>Here's the result of my code:</p>
  <pre><code>ex1.png 21.052631578947366
  ex2.png 0.0
  ex3.png 42.10526315789473
  </code></pre>
  <p><a href="https://i.stack.imgur.com/R1HQP.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/R1HQP.png" alt="enter image description here" /></a>
  <a href="https://i.stack.imgur.com/o0fZZ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/o0fZZ.png" alt="enter image description here" /></a>
  <a href="https://i.stack.imgur.com/Ysvtq.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Ysvtq.png" alt="enter image description here" /></a></p>
  <p>It does alright! It was able to tell that ex1 is similar and ex2 is not similar, however it states that ex3 is similar (even more similar than ex1). Any extra pre-processing or post-processing (maybe ml, assuming ml is actually useful) or just changes I can do to my method that can be done to keep only ex1 as similar and not ex3?</p>
  <p>(Note this score I create is something I found online. Not sure if it's an accurate way to go about it)</p>
  <p><strong>ADDED MORE EXAMPLES BELOW</strong></p>
  <p>Another set of examples:</p>
  <p>Here's what I am searching for</p>
  <p><a href="https://i.stack.imgur.com/NNWR5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/NNWR5.png" alt="enter image description here" /></a></p>
  <p>I want the above image to be similar to the middle and bottom images (NOTE: I rotate my target image by 45 degrees and compare it to the images below.)</p>
  <p>Feature matching (as stated in answers below) were useful in found similarity with the second image, but not the third image (Even after rotating it properly)</p>
  <p><a href="https://i.stack.imgur.com/kyTQE.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/kyTQE.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.stack.imgur.com/61aqC.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/61aqC.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.stack.imgur.com/hvXS5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/hvXS5.png" alt="enter image description here" /></a></p>