What is the name of this problem in Computer Vision?
This seems to be exactly scene description as described, for example, in Timor Kadir's thesis:


  Broadly speaking, the aim of scene description is to arrive at a set of descriptions of a real world scene which sufficiently capture the component parts of the scene, their positions, poses, motions and interactions. (p.&nbsp;3)