1. A method of rendering a synthetic object onto a user-worn display showing a user's view of a real world scene, the method comprising the steps of: capturing real world scene information using one or more user-worn sensors; producing a first pose estimation data set and a first depth data set based on at least a portion of the captured real world scene information, wherein the first pose estimation data set includes predicted inter-frame pose information for at least one camera of the one or more user-worn sensors to compensate for a data capture time lad between the at least one camera and at least one other user-worn sensor of the one or more user-worn sensors; receiving the synthetic object generated in accordance with the first pose estimation data set and the first depth data set; rendering the synthetic object onto the user-worn display in accordance with the first pose estimation data set and the first depth data set, thereby integrating the synthetic object into the user's view of the real world scene; producing a second pose estimation data set and second depth data set for a user-controlled device; and determining, based at least in part on the first pose estimation data set and the second pose estimation data set, whether a user-controlled device has interacted with the synthetic object and causing the synthetic object to react to the interaction based at least in part on the first pose estimation data and the second pose estimation data set.