If say func is GL_NEVER, the stencil test will always fail. Neither Color/Z-buffers are modified. The stencil buffer is modified as per glStencilOp fail. If say glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP) then GL_REPLACE takes place and stencilValue = (ref & stencilMask) // will become ref 2. Stencil Test Function passes/Depth Test Function fails: If say func is GL_ALWAYS, the stencil test will always pass, but depth test may fail. Neither Color/Z-buffer are modified. The stencil buffer is modified as per glStencilOp zfail. If say glStencilOp(GL_KEEP, GL_INCR, GL_KEEP) then GL_INCR takes place and stencilValue = (stencilValue+1) // will become 1 3. Stencil Function passes/Depth Function passes: If say func is GL_ALWAYS, the stencil test will always pass. If the depth test also passes. Both Color/Z-buffer are modified. The stencil buffer is modified as per glStencilOp zpass. If say, glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP) then Stencil values are not changed, only Color and Z-buffers are modified. Typically Stencil buffer is initialized by setting Z-buffer and color buffer masks to false. and then setting appropriate ref value to stencil buffer by failing the stencil test every time. // disable color and Z-buffers glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE); glDepthMask(GL_FALSE); glStencilFunc(GL_NEVER, 1, 0xFF); // never pass stencil test glStencilOp(GL_REPLACE, GL_KEEP, GL_KEEP); // replace stencil buffer values to ref=1 glStencilMask(0xFF); // stencil buffer free to write glClear(GL_STENCIL_BUFFER_BIT); // first clear stencil buffer by writing default stencil value (0) to all of stencil buffer. draw_stencil_shape(); // at stencil shape pixel locations in stencil buffer replace stencil buffer values to ref = 1 Now use the initialized stencil buffer and stencil test to write only in the locations where the stencil value is 1: // enable color and Z-buffers. glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE); glDepthMask(GL_TRUE); // no more modifying of stencil buffer on stencil and depth pass. glStencilMask(0x00); // can also be achieved by glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP); // stencil test: only pass stencil test at stencilValue == 1 (Assuming depth test would pass.) // and write actual content to depth and color buffer only at stencil shape locations. glStencilFunc(GL_EQUAL, 1, 0xFF); draw_actual_content(); See also Z-buffering (depth buffer) Shadow volume (a stencil buffer technique creating shadows) References Computer graphics 3D rendering