Projective texturing from many images Problem: I'm trying to dynamically texture a mesh of the real world generated by the Hololens over time using photos also captured from the Hololens, perhaps one photo per second. Each of the photos includes positional information of the Hololens from the time when the photo was taken, which allows me to project the images onto the existing mesh and figure out the UV mapping for each image.

In case of a single image, this seems fairly simple, as it is just standard projective texturing of the mesh. But I will be capturing a potentially endless stream of images over a span of time which will be taken from different positions and orientations. For acceptable performance, I cannot simply store and iterate through all photos and do projective texturing for each.

How can I perhaps generate a texture atlas &amp; accompanying UV mapping for the real world mesh and update it after each image?
Apologies in advance for the poor quality of this answer, but this sounds a little like what an ex-colleague was doing as part of his PhD. The "Free-Viewpoint video" papers listed at the bottom of https://www.researchgate.net/profile/James_Imber might be useful, or at least a starting point to find related work.