OpenGL: Rendering 2D shadows I've been working on a 2d shadow system for a school project, using this method to generate visibility polygons. However, I'm having issues figuring out how to draw those visibility polygons in a way that reveals everything drawn within them, rather than drawing over them like a normal sprite would.

From what I've been able to figure out, it seems like what I want to do is to draw the visibility polygons to some texture first, and then use that texture somehow when drawing the rest of the objects on the screen to get the shadow/light effect.

Unfortunately all the tutorials I've been able to find aren't very helpful. So, my questions are:

1) How do I tell OpenGL to render to a texture instead of the screen?

2) How do I use that texture to create the effect I want?
You don't need to render to texture to achieve an effect like this. Use stencil buffer.

The visibility polygon can be drawn using GL_TRIANGLE_FAN, if you start from the middle:



At the beginning of each frame:


Clear the stencil buffer (and enable stencil test)
Set the stencil function to always pass, and to replace the current value in the buffer to a greater than zero number (for example one)
Draw the visible part as a GL_TRIANGLE_FAN. If you have multiple characters/lights, draw the visibility polygon for each.


Now the stencil buffer contains 1 at visible locations and 0 where the character can't see.

Once you have a stencil buffer like this, you can control which objects you want to draw using glStencilFunc. Do not write to the stencil buffer, just use the stencil test to discard fragments which are visible/invisible.

If you want to draw a background which is visible everywhere, use GL_ALWAYS. For objects which are only displayed in the visible area use GL_EQUAL and 1. Similarly you can draw something (for example a dark quad) just in the invisible parts of the scene with GL_EQUAL and 0.

If it is not clear, how to use stencil buffers, check out this tutorial