   Shadow map alpha testing question [SOLVED].
  Hey guys...I've recently implemented realtime shadow maps into my game "Gang War", it's coming along very nicely, I'm just about done with 'em.  I'm rendering 2 shadow maps every frame, a global shadow map that covers the entire city, and a local shadow map that is focused on the camera, and I blend between the two.   Here are 2 videos showing how it looks in-game (you'll need DivX to watch these).... Anyways...I'm implementing alpha testing for fences, trees, etc.  It doesn't seem to work.  I was under the impression that shadow maps could easily support alpha testing in the shadows...ie: when rendering the scene from the light's perspective, just don't render texels that fail the alpha test.  Though it doesn't seem to work.  I've tried rendering the shadow map overlayed on screen and all parts that should fail the alpha test, and not be rendered, appear black (and cast a shadow).I'm explicitly enabling alpha testing right before rendering to the shadow maps, and setting the ALPHA_REF as well.  Alpha testing works fine in every other part of the scene, though it doesn't seem to work for the shadow maps, here is another screenshot showing the problem....the shadow of the fence isn't alpha tested :-/Is there something I'm missing?  Is there a special way to use alpha testing when rendering shadow maps?I'd appreciate any help/advice you guys might have to offer. - Dan[Edited by - dgreen02 on December 10, 2005 3:44:03 AM]
  It should work, assuming you still set the texture containing the alpha, and setup your colorop/alphaop (remember a stage must have both a color and alphaop, or both be set to disable, in order to be valid) or pixel shader to fetch from the texture.  I've done nVidia style shadow maps (ie: use depth buffer as a texture on ps.1.1 hardware) with alphatest and it worked as expected.  Also, make sure you're not trying to over optimize your shadow geometry by stripping out UVs.   ;  Hmmm....Yup, I'm fairly sure I'm doing all that stuff correctly.Here is another screenshot, this is the what the shadow map looks like when I sample the texture, instead of just outputting the depth in the shadow map pixel shader....and then display it...the shadow map is of D3DFMT_R32F format.The circled areas are the problematic parts...I'm using alpha tested imposters for distant street lights, and the fence doesn't appear correct either :-/;  Anybody have any ideas?  I'm still looking for a solution to this problem.  Any suggestions would be appreciated!The alpha testing happens after the pixel is sent from the shader, and before it's written to the surface, so the fact that I'm rendering to texture with a single channel shouldn't have anything to do with it, correct? - Dan   ;  It's getting a bit late and my brain is starting to turn off... but I'm sure I've read about this before [oh]Something to do with alpha testing still filling the depth buffer and only discarding the colour write.Provided I am correct... If I can find a source for that or remember the details I'll get back to you [smile]Jack   ;  Hi dgreen02,your game look cool ! :)Recent video cards does not support alpha testing with float rendertarget formats !If your shadowmap is D3DFMT_R32F you must emulate alpha-testing manuallyby killing pixels using texkill instruction.Petr S.;  [EDIT: Oops, too slow!]It is becuase alpha testing is not RT dependent, and most current GFX cards supporting FP render targets do not support post pixel shader blending operations (which includes alpha testing) when using them. I think there is either a cap to check for support or you can use CheckDeviceFormat or similar I think...It's a real pain in the ass!You can use texkill\clip() in your pixel shader though. One possibility is to render all opaque casters separately in a few large drawPrimitive calls (as you don't need their material info) and then render all alpha-testing casters with a pixel shader using texkill. This may get you some speed optimisations on some cards that take a different internal path if the pixel shader uses texkill.Hope this helps,Mike.   ;  I know, it is a bit off-topic (that is, not answering your question), but would you mind to explain your two shadow maps system a bit? I'm very curious! Thanks!   ;  That FP render targets thing sounds similar to what I was trying to remember earlier. The closest I could remember before reading that was that you had to emulate it using texkill/clip()...!Quote:Original post by roelI know, it is a bit off-topic (that is, not answering your question), but would you mind to explain your two shadow maps system a bit? I'm very curious! Thanks!Check out his developer journal - it's without a doubt one of the better ones. I forget when exactly, but he's talked about the various bits of technology used in his game/engine [smile]cheers,Jack   ;  Great...I appreciate the help guys!  I'm sure I'll be able to get it working now.  roel - I'll also be posting some info in my developers journal, as well as some videos showing the all the new stuff, sometime today, so check it out if you're interested :-DThanks again for your help guys. - Dan   