360 world based on different pictures We are modelling interiors in Blender one can walk through in the browser (three.js). You can get close to windows, but can't go outside. We are struggling with getting a good experience of the outside environment.

We basically use fisheye pictures as input, taken from each window. In three.js we now create one sphere around the full building on which we position the fisheye pictures so that they can be seen from the corresponding windows. In Blender this corresponds to a World with one 360 background picture.

There are multiple problems with this:


It is first of all difficult to manually position the pictures correctly for each window on that one background image.
Some views overlap each other. When positioning the fisheye pictures correctly at a window they overlap with the picture of a nearby window. One would need complex algorithms to merge these pictures.
Some views have a gap. This is the opposite of the previous problem. At some windows, one can see beyond the edges of the fisheye picture.
The shift in perspective when looking at a window and moving backwards is bad. Especially at windows starting at floor level. Nearby objects move away too much.
You get the feeling that the house is floating / positioned too high. This is caused because the house is centered in the sphere.


Using a sperical shape positioned around each window could possibly solve issues 2 and 3. It might even be more straightforward to create, thus solve problem 1.

But for problem 4 and 5 we think we might need a different shape than a pure sphere. Something that is more flat at the floor level, but spherical further away/in the sky. The challenge here, again, how does one convert fisheye pictures to this custom shape?

We are looking for some basic guidance/experiences here. How did you tackle this? Maybe a complete different approach?
Try an HDR background image. You can get several from here.

This is a single, 360º background image that also has the nice advantage of being able to light your scene!

An example:





This scene used absolutely no lights, just the background image. The shadow on the ground is provided by a shadowcatcher plane. (The particular HDR I used is public domain and can be found here.)