Beyond cross-eyed 3D (VR application)
I have finally found a great overview video on the topic from Richard Szeliski: "Reflections of Image-Based Modelling and Rendering"

The research started with stereo and depth estimation. The depth map can be used to "warp" image in 3D to give it some perceived depth. This does not create parallax effects though.

Another approach called Layered Depth Images used segmentation to several depth layers each having its own depth map. This allows parallax effect to occur, but extra care must be taken to matte foreground and background layers during rendering.

Fast 3D reconstruction of the sorrounding is possible using primitives (boxes, cylinders) matched to scene points. The 3D reconstruction is tedious and requires many images and extensive walk around.

Then, more image-based (rather than geometry-based) approach are presented, which are Lumigraphs and Light Fields. These allow some level of reconstruction of the plenoptic function and solve for rays crossing given viewing plane.

Finally, Concentric Mosaics is the no. 1 solution I have been looking for. A camera travelling around circle (tangential direction) is able to capture 3D plenoptic function of the scene. Each point thus can be viewed from different angles and the viewer can move inside the circle, see parallax, reflections, transparency and occlussion.

Extension is possible by adding support for rays in vertical direction - this will lead to capturing 4D plenoptic function from a camera moving on sphere (tangetial direction) or by using a spherical rig of many cameras.

Further advances in the field are expected with the introduction of plenoptic cameras (e.g. Lytro) that can capture the 4D plenoptic function directly. Some work on making plenoptic mosaics is already there (e.g. "Panorama Light Field Imaging").