title: OpenCV : 2d - 3d point correspondence using chessboard
body: <p>I'm facing a problem when I try to find the correspondences between a 3D point located on my chessboard and its position on the image taken by the camera.</p>
  
  <p>Here is what I do:</p>
  
  <p>I use the method </p>
  
  <pre><code>corners = cv.FindChessboardCorners(frameGray, patternSize, cv.CV_CALIB_CB_ADAPTIVE_THRESH + cv.CV_CALIB_CB_NORMALIZE_IMAGE)
  </code></pre>
  
  <p>which gives me the position of all the internal corners of the chessboard. Because I know their positions in the real world I would like to use further the solvePnP method to find the camera pose.</p>
  
  <p>However, I noticed that the "corner order" changes depending of the orientation of my chessboard. I want to draw a green circle on my "first" corner but that "first" corner is different depending on my chessboard orientation.</p>
  
  <p>Here are some screenshots to explain what I am talking about:</p>
  
  <p><img src="https://i.stack.imgur.com/A6pXm.png" alt="http://i.imgur.com/cYtUIDy.png">
    <img src="https://i.stack.imgur.com/XWHHm.png" alt="http://i.imgur.com/04wvzL7.png"></p>
  
  <p>And here is the full code:</p>
  
  <pre><code>import cv
  
  
  patternSize = (7,7)#interior number of corners, that is, points, where the black squares touch each other
  
  cv.NamedWindow("w1", cv.CV_WINDOW_AUTOSIZE)
  camera_index = 0
  capture = cv.CaptureFromCAM(camera_index)
  c = cv.WaitKey(5000)#wait for the camera to be initialized
  def repeat():
      global capture #declare as globals since we are assigning to them now
      global camera_index
      global frame
      frame = cv.QueryFrame(capture)  
      if frame:       
              cv.ShowImage("w1", frame)
              #convert to grayImage 
              frameGray = cv.CreateImage(cv.GetSize(frame), cv.IPL_DEPTH_8U, 1)
              cv.CvtColor(frame,frameGray,cv.CV_BGR2GRAY)
  
              corners = cv.FindChessboardCorners(frameGray, patternSize, cv.CV_CALIB_CB_ADAPTIVE_THRESH + cv.CV_CALIB_CB_NORMALIZE_IMAGE)
              if (len(corners[1]) == patternSize[0]*patternSize[1]):#if we found all the internal corners
                      #Refines the corner locations
                      #cv.FindCornerSubPix(frameGray,corners[1],(11,11),(-1,-1),(cv.CV_TERMCRIT_ITER | cv.CV_TERMCRIT_EPS, 10, 0.01))
                      point = int(corners[1][0][0]), int(corners[1][0][1])
                      print (point)
                      patternWasFound = corners[0]
                      #cv.DrawChessboardCorners(frame, patternSize, corners[1], patternWasFound)
                      cv.Circle(frame, point, 4, (0, 255, 0), -1)
                      cv.ShowImage("w1",frame)
              c = cv.WaitKey(10)
  while True:
      repeat()
  </code></pre>
  
  <p>As you can see I always take the same corner (int(corners<a href="https://i.stack.imgur.com/A6pXm.png" rel="nofollow noreferrer">1</a>[0][0]), int(corners<a href="https://i.stack.imgur.com/A6pXm.png" rel="nofollow noreferrer">1</a>[0]<a href="https://i.stack.imgur.com/A6pXm.png" rel="nofollow noreferrer">1</a>)), but it changes depending on my chessboard orientation. </p>
  
  <p>Is there anyone who knows why ? If the corner location changes, I can't know the corresponding real coordinates then :/</p>
  
  <p>Thank you !</p>