Mesh reconstruction using tracking data I searched around Blender SE, and I have seen no questions down this slant at all, so I'm going to ask it. 

Blender uses all the information necessary to successfully solve a track to re-position a camera in 3D space in relation to the tracked footage. 

My question is, since all of these "Control Points" get solved to some high degree of accuracy, and become a known distance and angle to the camera, is there any quick way to actually build a mesh from the video track once it is solved (allowing the rest of the texture to become auto-trackable based on commonality between frames - maybe something similar to how microsoft kinect gets used for 3D scanning, but as a post process after the solve of course)?



To maybe add more references of my thought process into something (at hopefully the most simplistic level) most could follow (as you jump into my rabbit hole of theoretical thinking), it would be something like the following image portrays.



If I imagine this fast wiggle as an accelerated fast-forward/rewind view of the footage &amp; solve, my own brain is solving the 3D of this image (even if I close one eye - unlike stereo view). 

This leads me to believe that Inverse to a stereo camera (such as a kinect/artec snanner/ etc... - which uses at least two camera's to form a triangle to millions of points in space, and solve their positions) you would have multiple basically solved locations in the space triangulating back to at least a single camera (eg. my one eye open that is pretty stationary OR both eyes open which helps average the solve in my brain I guess, Ultimately the Blender Camera(s)). This effect works so well, because it acts as a POV (persistence of vision - not referring to point of view) to the brain. It just so happens that Blender has this information in the next &amp; previous frame locations in the form of pixel data of the footage, so what's the technical difference in terms of visual memory/persistence?

Also this image is very clean in terms of focal subject and little occlusion, except for at the very perimeter of the sphere. All the rest of the Image Texture is clean and trackable (hopefully with some sort of automatic help in the future).

I've tried this visually/manually from two images, and got some pretty amazing results, but it requires manipulating the mesh in such a way to where the topology in true 3D space &amp; the UV's in 2D space would form the solve to look good.
There is currently no way to reconstruct a more detailed mesh between the camera-solved Control Points from within blender. Thats because the Camera Solver does not look at the image data at all, It only deals with the marker positions. The Control Point-Tracker is the one who finds commonality between frames. So you could just add more Control Points, but you'll of course never get a z-value for every pixel of the frame.

So I see this two ways of achieving, what I think you want:

Manual Scene Reconstruction - this works well with rectangular scenes like rooms, hallways, roads, houses

Blender Movie Clip Editor offers a few basic tools for scene orientation after camera solution: automatic origin, floor or wall plane setup from selected tracks. With this you can line up Blenders coordinate-grid with rectangular objects in the movie clip. In a camera-viewport with the movie-clip enabled as background its easy now to manually reconstruct rectangular objects via extrude on the axes x,y or z. Also, the BLAM Blender addon may be helpful with this process. It also features camera solution from one single image instead of a movie clip.
In the end you can texture the mesh using the UV-projection modifier to project the movie onto the mesh.

Foto Series Reconstruction Workflow - also works with non-rectangular scenes and objects, like reliefs, uneven terrain

Extract ca. 25 frames with significant parallax from your clip (e.g. take every 10th frame), then...

Option A:
Use the free 123D Catch-App from Autodesk to generate the model and export it to Blender.

Option B:
Follow this video tutorial, which describes the workflow of reconstructing and texturing a mesh from a series of fotos using the free programs VisualSFM and Meshlab, and exporting the model to Blender.
VisualSFM is used to reconstruct a 3d-point-cloud from a series of fotos, with Meshlab you approximate a mesh from the points and optionally create a UV-Map with projected parts of the fotos as texture. From there the model can be exported to Blender.