1. A product comprising a processing system containing at least one processor that has a configuration that causes at least the following to occur, or a non-transitory, tangible, computer-readable storage medium containing a program of instructions that causes a computer system running the program of instructions to cause at least the following to occur: receive a stream of video images of a scene, each image having been captured by an image capture device while located at a particular position and having a particular orientation, at least two of the images having been captured by the image capture device while at different locations; after receiving each image and before processing the next image, generate information indicative of the position and orientation of the image capture device at the time of capturing the image, the generate information including: identifying distinguishable image feature points in the image; determining a coordinate for each identified image feature point; for each identified image feature point, attempting to identify one or more distinguishable model feature points in a three dimensional (3D) model of at least a portion of the scene that appears likely to correspond to the identified image feature point; and finding each of the following that, in combination, produce a consistent projection transformation of the 3D model onto the image: a subset of identified image feature points for which one or more corresponding model feature points were identified; and for each image feature point that has multiple likely corresponding model feature points, one of the corresponding model feature points.