Python Opencv SolvePnP yields wrong translation vector
Question: I am attempting to calibrate and find the location and rotation of a single
virtual camera in Blender 3d using homography. I am using Blender so that I
can double check my results before I move on to the real world where that is
more difficult. I rendered ten pictures of a chess board in various locations
and rotations in the view of my stationary camera. With opencv's python, I
used cv2.calibrateCamera to find the intrinsic matrix from the detected
corners of the chess board in the ten images and then used that in
cv2.solvePnP to find the extrinsic parameters(translation and rotation).
However, though the estimated parameters were close to the actual ones, there
is something fishy going on. My initial estimation of the translation was
(-0.11205481,-0.0490256,8.13892491). The actual location was (0,0,8.07105).
Pretty close right? But when I moved and rotated the camera slightly and
rerendered the images, the estimated translation became farther off.
Estimated: (-0.15933154,0.13367286,9.34058867). Actual:
(-1.7918,-1.51073,9.76597). The Z value is close, but the X and the Y are not.
I am utterly confused. If anybody can help me sort through this, I would be
highly grateful. Here is the code (it's based off of the python2 calibrate
example supplied with opencv):

    
    
    #imports left out
    USAGE = '''
    USAGE: calib.py [--save <filename>] [--debug <output path>] [--square_size] [<image mask>]
    '''   
    
    args, img_mask = getopt.getopt(sys.argv[1:], '', ['save=', 'debug=', 'square_size='])
    args = dict(args)
    try: img_mask = img_mask[0]
    except: img_mask = '../cpp/0*.png'
    img_names = glob(img_mask)
    debug_dir = args.get('--debug')
    square_size = float(args.get('--square_size', 1.0))
    
    pattern_size = (5, 8)
    pattern_points = np.zeros( (np.prod(pattern_size), 3), np.float32 )
    pattern_points[:,:2] = np.indices(pattern_size).T.reshape(-1, 2)
    pattern_points *= square_size
    
    obj_points = []
    img_points = []
    h, w = 0, 0
    count = 0
    for fn in img_names:
        print 'processing %s...' % fn,
        img = cv2.imread(fn, 0)
        h, w = img.shape[:2]
        found, corners = cv2.findChessboardCorners(img, pattern_size)        
    
        if found:
            if count == 0:
                #corners first is a list of the image points for just the first image.
                #This is the image I know the object points for and use in solvePnP
                corners_first =  []
                for val in corners:
                    corners_first.append(val[0])                
                np_corners_first = np.asarray(corners_first,np.float64)                
            count+=1
            term = ( cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_COUNT, 30, 0.1 )
            cv2.cornerSubPix(img, corners, (5, 5), (-1, -1), term)
        if debug_dir:
            vis = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
            cv2.drawChessboardCorners(vis, pattern_size, corners, found)
            path, name, ext = splitfn(fn)
            cv2.imwrite('%s/%s_chess.bmp' % (debug_dir, name), vis)
        if not found:
            print 'chessboard not found'
            continue
        img_points.append(corners.reshape(-1, 2))
        obj_points.append(pattern_points)        
    
        print 'ok'
    
    rms, camera_matrix, dist_coefs, rvecs, tvecs = cv2.calibrateCamera(obj_points, img_points, (w, h))
    print "RMS:", rms
    print "camera matrix:\n", camera_matrix
    print "distortion coefficients: ", dist_coefs.ravel()    
    cv2.destroyAllWindows()    
    
    np_xyz = np.array(xyz,np.float64).T #xyz list is from file. Not shown here for brevity
    camera_matrix2 = np.asarray(camera_matrix,np.float64)
    np_dist_coefs = np.asarray(dist_coefs[:,:],np.float64)    
    
    found,rvecs_new,tvecs_new = cv2.solvePnP(np_xyz, np_corners_first,camera_matrix2,np_dist_coefs)
    
    np_rodrigues = np.asarray(rvecs_new[:,:],np.float64)
    print np_rodrigues.shape
    rot_matrix = cv2.Rodrigues(np_rodrigues)[0]
    
    def rot_matrix_to_euler(R):
        y_rot = asin(R[2][0]) 
        x_rot = acos(R[2][2]/cos(y_rot))    
        z_rot = acos(R[0][0]/cos(y_rot))
        y_rot_angle = y_rot *(180/pi)
        x_rot_angle = x_rot *(180/pi)
        z_rot_angle = z_rot *(180/pi)        
        return x_rot_angle,y_rot_angle,z_rot_angle
    
    print "Euler_rotation = ",rot_matrix_to_euler(rot_matrix)
    print "Translation_Matrix = ", tvecs_new
    

Thank you so much


Answer: I think you may be thinking of `tvecs_new` as the camera position. Slightly
confusingly that is not the case! In fact its the position of the world origin
in camera co-ords. To get the camera pose in the object/world co-ords, I
believe you need to do:

    
    
    `-np.matrix(rotation_matrix).T * np.matrix(tvecs_new)`
    

And you can get the Euler angles using `cv2.decomposeProjectionMatrix(P)[-1]`
where `P` is the `[r|t]` 3 by 4 extrinsic matrix.

I found [this](http://ksimek.github.com/2012/08/22/extrinsic/) to be a pretty
good article about the intrinsics and extrinsics...

