How to project a texture on objects in an (fSpy-imported) scene? With the amazing tool fSpy I wanted to reconstruct the interior of a house. I was able to import the fSpy project into Blender, do some basic modelling and UV &quot;Project from view&quot; seemed to work fine.
However, some parts of the image are not correctly UV projected, see image below. The corner of the table in the lower left is not projected to, which is to be expected outside the camera view, but not within camera boundaries! Additionally, when looking at the tray on top of the table, the background image does not align with the UV projected lower portion of the tray.

Question 1: In the 3D view (right), why does the UV projection not line up with the preview in the UV editor (left)?
Question 2: How to make it work?


Edit: I was able to answer the questions myself, see below. After some more trial and error I found an even better solution than described below. I will notes on that later this week!


Additional notes
I suspect this UV coordinates that are off boundaries (x&lt;0 or y&lt;0) are somehow being automatically translated somehow, but I don't really know.
I made two small additional changes to better see what's going on. Cutting out those changes did not solve the problem.

In the shader nodes editor, I have set the image texture from Repeat to Clip, so image on the right the problematic area is black and not couch-coloured.
For the table object I added a subdivision modifier set to Simple (not Catmull-Clark). Without it the deformation on the table-top would be based on two triangles with even works deformation.

Related questions

UV unwrapping problem with image mapping
Project from view so that the UVs match the background image
Why am I getting UV distortion on a flat surface?
I was able to solve the problem after projecting more objects in the scene. The UV coordinates are mapped correctly into the scene from the point of view of a flat surface (like the table top), but because of that not from the point of view of the camera. It does not have anything to do with coordinates being &quot;off bounds&quot;.
Imagine a ladder lying on the floor. The rungs have equal distances, but seen &quot;in perspective&quot; the distance between two rungs is getting smaller the further away they are. This non-linearity in perspective causes the linear UV projection on the table top to stretch unintendedly for big faces.
The problem can be solved by having more UV vertex information, i.e. more vertices in the mesh. This way Blender will not stretch the image on a projected face linearly, but follow the non-linearity specified by the additional vertices. Technically this will still stretch the projected image linearly for each face (i.e. not as intended), but this won't be noticable, if the faces are small enough.