<p>I saw a question on reverse projecting 4 2D points to derive the corners of a rectangle in 3D space.  I have a kind of more general version of the same problem:</p>

<p>Given either a focal length (which can be solved to produce arcseconds / pixel) or the intrinsic camera matrix (a 3x2 matrix that defines the properties of the pinhole camera model being used - it's directly related to focal length), compute the camera ray that goes through each pixel.</p>

<p>I'd like to take a series of frames, derive the candidate light rays from each frame, and use some sort of iterative solving approach to derive the camera pose from each frame (given a sufficiently large sample, of course)... All of that is really just massively-parallel implementations of a generalized Hough algorithm... it's getting the candidate rays in the first place that I'm having the problem with...</p>

[{'answer_id': 76807, 'author': 'Matt Cruikshank', 'author_id': 8643, 'author_profile': 'https://Stackoverflow.com/users/8643', 'pm_score': 1, 'selected': False, 'text': "<p>A friend of mine found the source code from a university for the camera matching in PhotoSynth.  I'd Google around for it, if I were you.</p>\n"}, {'answer_id': 76995, 'author': 'Community', 'author_id': -1, 'author_profile': 'https://Stackoverflow.com/users/-1', 'pm_score': 0, 'selected': False, 'text': '<p>That\'s a good suggestion... and I will definitely look into it (photosynth kind of resparked my interest in this subject - but I\'ve been working on it for months for robochamps) - but it\'s a sparse implementation - it looks for "good" features (points in the image that should be easily identifiable in other views of the same image), and while I certainly plan to score each match based on how good the feature it\'s matching is, I want the full dense algorithm to derive every pixel... or should I say voxel lol?</p>\n'}, {'answer_id': 84026, 'author': 'Matt Cruikshank', 'author_id': 8643, 'author_profile': 'https://Stackoverflow.com/users/8643', 'pm_score': 0, 'selected': False, 'text': "<p>After a little poking around, isn't it the extrinsic matrix that tells you where the camera actually is in 3-space?</p>\n\n<p>I worked at a company that did a lot of this, but I always used the tools that the algorithm guys wrote.  :)</p>\n"}]
['https://Stackoverflow.com/questions/76718', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/-1/']