1. A non-transitory computer readable memory medium storing programming instructions executable by a processor to: receive, from at least one camera, one or more images of a physical environment of a user of a three-dimensional (3D) display system; determine a data imagery map of the physical environment based at least in part on the one or more images; receive instructions to render a virtual 3D scene on a display of the 3D display system; and incorporate the data imagery map of the physical environment into the virtual 3D scene, thereby incorporating physical world conditions into virtual world imagery.