   stencil buffer setup
  Hi!I am having a problem with the stencil buffer (directX 8). My plan is as follows; i have set up the stencil buffer as to first render a polygon which sets values in the buffer an as such acts as a mask for later rendering. Then i change the settings to render a scene which should be rendered onto all pixels except the ones masked out by the first polygon. Then i change the settings to render a scene that should only be drawn on the masked out pixels. However, the code i am using does not pull this off, and i do not know what is wrong! the code i am using is as follows:...// set up the stencil bufferGraphics()->GetDevice()->SetRenderState (D3DRS_STENCILENABLE, TRUE);Graphics()->GetDevice()->SetRenderState (D3DRS_STENCILFUNC, D3DCMP_ALWAYS);Graphics()->GetDevice()->SetRenderState (D3DRS_STENCILPASS, D3DSTENCILOP_INCR);Graphics()->GetDevice()->SetRenderState (D3DRS_ALPHAFUNC, D3DCMP_NEVER);Graphics()->GetDevice()->SetRenderState (D3DRS_ALPHATESTENABLE, TRUE);// Render a polygon here-> very personalized code - i know it works normally!// set up the stencil buffer for first sceneGraphics()->GetDevice()->SetRenderState (D3DRS_ALPHAFUNC, D3DCMP_ALWAYS);Graphics()->GetDevice()->SetRenderState (D3DRS_ALPHATESTENABLE, FALSE);Graphics()->GetDevice()->SetRenderState (D3DRS_STENCILFUNC, D3DCMP_EQUAL);Graphics()->GetDevice()->SetRenderState (D3DRS_STENCILPASS, D3DSTENCILOP_KEEP);Graphics()->GetDevice()->SetRenderState (D3DRS_STENCILREF, 0);// draw the first scene-> works fine ordinarily // set up the stencil buffer for second sceneGraphics()->GetDevice()->SetRenderState (D3DRS_STENCILREF, 1);// draw the second scene-> also works fine originally!...Am i setting up the stencil buffer correctly! The problem is (i think) that no mask is created!I appreciate all help!regardsKrokus 
  1) Are you chosing an "AutoDepthStencilFormat" in your present parameters that actually has stencil bits ?, i.e. something like D3DFMT_D15S1 or D3DFMT_D24S8 ?2) If you have done the above, and are running on an nVidia card: if you''re in windowed mode, make sure the desktop and backbuffer is 32bit. If you''re in fullscreen mode, make sure your backbuffer is 32bit.3) Are you remembering to clear the stencil buffer in your Clear() call. i.e. D3DCLEAR_STENCIL ?It''s probably worth comparing your setup to one of the similar stencil buffer samples in the SDK. --Simon O''Connorex  -Creative AsylumProgrammer &Microsoft MVP;  Thanks for the swift reply Simon.As to your comments:1) yes, i am using D3DFMT_D24S8;2) i am actually using an nvidia card, and i have double-checked: i am using a 32 bit backbuffer.3) I am erasing the stencil buffer in the Clear() call.It is a good idea to check the setup in the SDK, but i have found the material on stencil buffers ther very very poor. I have also searched the net, but come up with nothing than can help me.i am very grateful to anyone who can help!Krokus;  ok fellas, i found the problem! However, i do not understand it: the code i listed above works fine if i remove the 4 lines of code that turns on and off the alpha test.Regards, Krokus. ;  Ah, I should have looked at that closer last night.Are you trying to use the alpha test to not render the pixel to the render target?. That's the way it looks with D3DRS_ALPHAFUNC as D3DCMP_NEVER (i.e. always fail the test/reject the pixel).IIRC alpha testing usually always happens BEFORE the stencil buffer is written to. So if you're always failing that test (D3DCMP_NEVER), the pixel will never be written to the stencil buffer.If you want to disable writes to the render target, there are much better, and much more established ways of doing that:1) Use D3DRS_COLORWRITEENABLE (when the D3DPMISCCAPS_COLORWRITEENABLE caps bit is set in PrimitiveMiscCaps)2) Enable alpha blending, set a mode of D3DBLEND_ZERO:D3DBLEND_ONE and disable Z writes. That's supported on a all drivers. Although it may seem wasteful, it was the typical method used to do exactly what you're trying to do before D3DRS_COLORWRITEENABLE was introduced into the API. So most drivers actually detect that blend and disable colour writes rather than actually performing the blend. If you look at the StencilMirror sample in the SDK you'll see this method being used.--Simon O'Connorex    -Creative AsylumProgrammer &Microsoft MVP [edited by - s1ca on August 2, 2003 7:50:20 AM];  Cheers Simon!That was useful! thanks!-Krokus 