title: ARKit - Simulating the change of ARCamera
body: <p>I am interested in simulating the projection of the corners of an <code>SCNPlane</code> from different positions and orientations of <code>ARCamera</code>. </p>
  
  <p>As expected, ARKit doesn't allow us to manipulate the position and orientation of the ARCamera. This means that I need to capture the transform of the camera with respect to the world, manipulate it (translate and/or rotate) and then use the intrinsic matrix to get the projected point on the image plane.</p>
  
  <p>The question is, how can I simulate different ARCamera poses? How would you reproduce the projections to the image plane by relying on the ARCamera pose?</p>
  
  <p><strong>Approach 1</strong></p>
  
  <p><code>K*C*P</code>, where <code>K</code> is the intrinsic matrix, <code>C</code> the transform of the camera with respect to the world and  <code>P</code> is the point we want to project expressed in world coordinates. I am surprised to see that this approach doesn't work - I am calling the native function <code>projectPoint</code> and then calling my custom approach and I am not getting the same results. </p>
  
  <pre><code>static func projectPointCustom(_ point: SCNVector3, sceneView: ARSCNView) -&gt; SCNVector3 {
          var p = SCNVector3()
          guard let camera = sceneView.session.currentFrame?.camera else { return SCNVector3()}
          let pCam = camera.transform * simd_float4(Float(point.x), Float(point.y), Float(point.z), 1.0)
  
          print("pCam = \(pCam)")
          p = SCNVector3(pCam.x/pCam.w, pCam.y/pCam.w, pCam.z/pCam.w)
          let res2 = camera.intrinsics * simd_float3(p)
  
          p = SCNVector3(res2)
          // let res = camera.projectionMatrix(for: .portrait, viewportSize: sceneView.bounds.size, zNear: 0.001, zFar: 1000.0) * simd_float4(Float(point.x), Float(point.y), Float(point.z), 1.0)
          // p = SCNVector3(res.x, res.y, res.z)
          return p
      }
  </code></pre>
  
  <p><strong>Approach 2</strong></p>
  
  <p>Using <code>viewMatrix</code> - per Apple's documentation, <code>viewMatrix</code> returns a transform matrix for converting from world space to camera space. So, convert point <code>P</code> to camera space, then use intrinsics to see where it hits in the image plane. Also doesn't work. </p>
  
  <pre><code>static func projectPointCustom(_ point: SCNVector3, sceneView: ARSCNView) -&gt; SCNVector3 {
          var p = SCNVector3()
          guard let camera = sceneView.session.currentFrame?.camera else { return SCNVector3()}
          let pCam = camera.viewMatrix(for: .portrait) * simd_float4(Float(point.x), Float(point.y), Float(point.z), 1.0)
  
          print("pCam = \(pCam)")
          p = SCNVector3(pCam.x/pCam.w, pCam.y/pCam.w, pCam.z/pCam.w)
          let res2 = camera.intrinsics * simd_float3(p)
          //let res2 = simd_float3(p)
          p = SCNVector3(res2)
          return p
      }
  </code></pre>