Screen capture of game video I'd like to "hook" into a running game, say Mario Bros, and capture each frame rendered... saving that frame to an image file. A good example of something similar is FRAPS. --Note: I don't want to capture the whole screen/desktop. I want to capture a targeted window.

I have had a look at OBS (Open Broadcasting Software) but it is not particularly fast. Don't get me wrong, it's great software, but unfortunately there is no/poor documentation, making a massive project written in c and c++ nearly inaccessible for a new-to-c++ programmer.

I've also had a look at GamingAnywhere, but unfortunately, I am unable to get it to work, there is very little/no documentation, runs in VS2010 only and is messy (with poor variable naming). However, it is a research project and so it is understandably undocumented and messy.

I know that this can be done with OpenGL, GDI and with Direct3D, but I am unable to find some good examples on the net. 

I read that glReadlPixels (using OpenGL) can be used and I've read the documentation, but the post mentioned nothing about hooking into a running game/application graphics.

Questions:


Can I hook into the graphics of a game developed with OpenGL, using,
say, Direct3D? Does the library used for hooking have to be the same
as the one used by the game?
How do I hook into the game's rendered frames so that I can output
those frames to image files or to a video file? (Just some links or
brief explanation of what I need to do would be great)
BackBuffer - I read that it is very fast accessing the BackBuffer to
retrieve the frames. Does someone have an example for me on how to
do this with the latest libraries? I have found that most examples
are out of date.
For my purposes, is there any clearly "this is faster than that"? What I mean is, would, say, OpenGL, be faster for my purposes? 


If someone knows of an open-source project (that does essentially what I need) that is actively developed and well documented, I'd love to know about it.
It is possible to do what you describe, but I'm afraid it is not a trivial process. Actually, this will be very tied to the Operating Systems you are targeting and possibly also requiring specific tweaks for the given game/application.

I would start looking into DLL injection techniques, which should allow, for instance, intercepting calls to the Direct3D (Windows) or OpenGL APIs, which you can then use to copy de framebuffers of the application. A quick Internet search turns up this and this with detailed explanation. 

I once wrote a small-scale OpenGL interceptor on MacOS by injecting a custom shared module, but it was under very controlled circumstances and making a lot of assumptions, such as having root privileges in the system, so for a production tool, it would be a lot more complicated. You can find another very interesting project here about intercepting the D3D API to install an automated AI bot on Starcraft, which should touch on very similar concepts to what you are looking for.

Once you manage to intercept the underlying rendering API, you could then perhaps hook your code into each SwapBuffers or equivalent call to just copy the previous framebuffer before the swap, then save it (this is where something like glReadPixels would come into play).

Copying and saving the framebuffer in an efficient manner is also challenging. The easy way is to just dump each frame as an uncompressed RGB image, but then you'll wind up with hundreds of gigabytes of data for just a couple of minutes of gameplay, so unless you have a nice HDD array sitting at the corner of your table ;), you'll need to look into compressing the frames somehow. 

The downside of compressing the frames now is that it takes a lot of processing, so a naive approach might turn the once interactive application you are trying to record into an interactive slide-show :(. So at this stage you'd have to start thinking about optimizations.

Don't know of any projects out there providing a library for efficient frame capture for games, but this would certainly be something nice to have! I think one thing that might be holding such project back is, like I mentioned at the beginning, that this is a very system dependent thing, so cases of use will most likely be limited to a single platform or OS.