title: OpenCV rotation (Rodrigues) and translation vectors for positioning 3D object in Unity3D
body: <p>I'm using "OpenCV for Unity3d" asset (it's the same OpenCV package for Java but translated to C# for Unity3d) in order to create an Augmented Reality application for my MSc Thesis (Computer Science). </p>
  
  <p>So far, I'm able to detect an object from video frames using ORB feature detector and also I can find the 3D-to-2D relation using OpenCV's SolvePnP method (I did the camera calibration as well). 
  From that method I'm getting the Translation and Rotation vectors. The problem occurs at the augmentation stage where I have to show a 3d object as a virtual object and update its position and rotation at each frame. OpenCV returns Rodrigues Rotation matrix, but Unity3d works with Quaternion rotation so I'm updating object's position and rotation wrong and I can't figure it out how to implement the conversion forumla (from Rodrigues to Quaternion). </p>
  
  <p>Getting the rvec and tvec:</p>
  
  <pre><code>    Mat rvec = new Mat();
      Mat tvec = new Mat();
      Mat rotationMatrix = new Mat ();
  
      Calib3d.solvePnP (object_world_corners, scene_flat_corners, CalibrationMatrix, DistortionCoefficientsMatrix, rvec, tvec);
      Calib3d.Rodrigues (rvec, rotationMatrix);
  </code></pre>
  
  <p>Updating the position of the virtual object:</p>
  
  <pre><code>    Vector3 objPosition = new Vector3 (); 
      objPosition.x = (model.transform.position.x + (float)tvec.get (0, 0)[0]);
      objPosition.y = (model.transform.position.y + (float)tvec.get (1, 0)[0]);
      objPosition.z = (model.transform.position.z - (float)tvec.get (2, 0)[0]);
      model.transform.position = objPosition;
  </code></pre>
  
  <p>I have a minus sign for the Z axis because when you convert OpenCV's to Unty3d's system coordinate you must invert the Z axis (I checked the system coordinates by myself).</p>
  
  <p>Unity3d's Coordinate System (Green is Y, Red is X and Blue is Z) :</p>
  
  <p><a href="https://i.stack.imgur.com/mgKmF.png" rel="noreferrer"><img src="https://i.stack.imgur.com/mgKmF.png" alt="enter image description here"></a></p>
  
  <p>OpenCV's Coordinate System:</p>
  
  <p><a href="https://i.stack.imgur.com/GsrKU.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/GsrKU.jpg" alt="enter image description here"></a></p>
  
  <p>In addition I did the same thing for the rotation matrix and I updated the virtual object's rotation. </p>
  
  <p>p.s I found a similar question but the guy who asked for it he did not post clearly the solution.</p>
  
  <p>Thanks!</p>