Rendering 3-D point clouds obtained from depth maps as a 2.5D video Suppose I have code to generate a bunch of frames, each of which is a depth map(I have access to the matrices which represent depth) which in turn is used to generate a point cloud. One frame would look like this:



In short I have a point cloud corresponding to each frame. How would I generate a video where each point cloud in a frame is meshed together? 

Are there frameworks which handle this kind of data? I am assuming OpenGL should help but I am not sure if it can do this.
Depending on your application, you could use image-based solutions that are much cheaper than meshing a uniformly sampled set of points. For example, you could use the depth map as a height map.

There are many techniques that allow you to render it in an efficient way, the more realistic the more difficult to code as a shader (but not too much). From easiest to harder:


height maps
normal maps
displacement mapping
parallax mapping
geometry shaders to recreate the actual mesh from a set of primitives and the depth map
and so on.