   Newbie needs help with Cg shadow mapping
  OK folks, I'm really sorry for bothering you with such an old story, but I am desperately trying to bring my Cg/OpenGL-Code to work in terms of shadow mapping. I have already produced a depth map with OpenGL and it is nicely applied as a texture on a simple quad by my Cg fragment shader (for debugging purposes). This little "render to texture"-experiment works fine. The next step is to perform the necessary depth tests for shadowing the scene. And this really gives me the creeps. I have generated the OpenGL Texture Matrix and gave it to the vertex shader, the shadow map is already available in the fragment shader. The problem is, that the texture coordinates generated via "mul(TexTransform, position)" are completely nonsense and my shadow is ... well, non-existent. Every single fragment passes the depth test so it is fully lit. Here's some code...(edit: i finally figured out how to produces those neat scrollpanes for sourcecode :)####### Vertex shader: ###########struct myVertex {float4 position  : POSITION;float4 color0    : COLOR0;float2 texcoord0 : TEXCOORD0;float3 normal : NORMAL;};struct fragment{float4 position  : POSITION;//screen positionfloat4 color0    : COLOR0;float4 color1    : COLOR1;//lightingfloat2 texcoord0 : TEXCOORD0;float4 SMTexCoord: TEXCOORD1;float3 normal : TEXCOORD2;};fragment main( myVertex IN, uniform float4x4 WorldViewProj,//transforms the vertex position into homogenous clip-space. uniform float4x4 WorldViewIT,//transforms (normal) vectors into eye space uniform float4x4 WorldView, uniform float4x4 TexTransform, uniform float3 LightVec){fragment OUT;float4 v = float4(IN.position.x,    IN.position.y,  IN.position.z,  1.0f);//Transform The Vertex Position Into Homogenous Clip-Space (Required)OUT.position  = mul(WorldViewProj, v);OUT.color0    = IN.color0;OUT.texcoord0 = IN.texcoord0;//Transform the normal from object-space to eye-spacefloat3 WorldNormal = normalize(mul((float3x3)WorldViewIT, IN.normal));OUT.normal = WorldNormal;float3 pointToLightVec = normalize(LightVec - IN.position.xyz);float3 pointToLightVecWorld = normalize(mul((float3x3)WorldViewIT,pointToLightVec));float ldotn = max(dot(pointToLightVecWorld,WorldNormal),0.0);OUT.color1.rgba = ldotn.xxxx;OUT.SMTexCoord = mul(TexTransform,v);return OUT;}####### Pixel shader: ###########struct fragment{float4 position  : POSITION;//screen positionfloat4 color0    : COLOR0;float4 color1    : COLOR1;//lightingfloat2 texcoord0 : TEXCOORD0;float4 SMTexCoord: TEXCOORD1;float3 normal : TEXCOORD2;};struct pixel{float4 color : COLOR;};float3 bumpit(float x, float y, float bias){float3 result;result.x = sin(bias*x);result.y = sin(bias*y);result.z = cos(bias*x)*cos(bias*y);return normalize(result)*0.5;}pixel main( fragment IN,uniform sampler2D testTexture : TEXUNIT0,  uniform sampler2D shadowMap   : TEXUNIT1,  uniform float4 light,  uniform float doBump,  uniform float doTexturing){pixel OUT;float4 localcolor = IN.color0;if (doTexturing &gt; 0.0){localcolor = tex2D(testTexture, IN.texcoord0.xy);}float4 mynormal;mynormal.w = 0;float4 mylight = light;mylight.w = 0;if (doBump &gt; 0.0){mynormal.xyz = IN.normal + bumpit(IN.texcoord0.x, IN.texcoord0.y, 40);localcolor = localcolor * abs(dot(normalize(mylight),normalize(mynormal)));}else{mynormal.xyz = IN.normal;}//shadowing (this is not working...):float4 depth = tex2Dproj(shadowMap,IN.SMTexCoord);float test = IN.SMTexCoord.z / IN.SMTexCoord.w;float shadow;shadow = (test &lt; depth.x) ? 1.0 : 0.0;OUT.color = localcolor * shadow * IN.color1;return OUT;}Thanks in advance for every piece of advice!(And sorry for my bad english, I'm out of practice...)[Edited by - NinjaSquirrel on December 12, 2005 3:38:27 AM]
  Are you sure tcGen is set up correctly?As far as I remember, the basic fragment program math is just fine.Please edit your post using code tags, everything looks way too ugly right now. I tried to read the code but after some effort I couldn't carry on.In the meanwhile, here's how I did it. Tell me if you're doing this.Make sure tcGen for shadow projection is set up accordingly. This basically requires simple projective texturing which is just a matrix multiply with the correct matrix.The matrix is basically a per-light MVP (I remember I had some troubles obtaining this but I don't remember why). Light's eye-space will naturally go in range [-1, -1] [1, 1] with (0,0) at center screen (like normal eye space after all) so there's some simple adjustments to be made.Check out projecting a normal texture. If it's correctly projected then you're almost ok.What will happen is that using two different xforms you end up in having both the real eye position and the eye position in light's eye space, which is then linearly interpolated. The z will come out from that.If something is wrong, I would point my finger at the vertex shader but it's quite different from what I was using and I'm not really sure of its meanings. I originally wrote it in ARB_vp and yours is something more lengthy...Is this what you believe you're doing?;Quote:Are you sure tcGen is set up correctly?Well, I am not really sure if am missing something here. To test my shadow map generation, I glued the shadow map as an ordinary texture to a quad and it shows what you'd expect: a gray-scale image with the scene rendered from the light's point of view. With that successfully working, I suppose my texture coordinates for that quad are correct. But I have to admit that I am deeply troubled with 'projecting' a texture onto a scene (see below)...The shadow map generation is done like this: I position the camera at the light position, render the scene, store the aquired data in my DEPTH_COMPONENT-texture, switch back to my "real" camera position and render that scene again. My OpenGL-Code for that is (well, I actually have stolen most of this from a tutorial...)://##### First render pass - from light's point of viewglClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);glMatrixMode(GL_PROJECTION);glLoadMatrixf(lightProjectionMatrix);glMatrixMode(GL_MODELVIEW);glLoadMatrixf(lightViewMatrix);//Use viewport the same size as the shadow mapglViewport(0, 0, shadowMapSize, shadowMapSize);//Draw back faces into the shadow mapglEnable(GL_CULL_FACE);glCullFace(GL_FRONT);//Disable color writes, and use flat shading for speedglShadeModel(GL_FLAT);glColorMask(0, 0, 0, 0);//Draw the scenemakeLighting();makeGeometry();//Read the depth buffer into the shadow map texture.//Since we have bound a DEPTH_COMPONENT texture,//the data read will automatically come from the depth bufferglBindTexture(GL_TEXTURE_2D, shadowMapTexture);glCopyTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 0, 0,shadowMapSize, shadowMapSize);//Restore the GL state changes of this render pass:glDisable(GL_CULL_FACE);glCullFace(GL_BACK);glShadeModel(GL_SMOOTH);glColorMask(1, 1, 1, 1);//##### End of first render pass - from light's point of view//update Transformation matrices:updateMatrices();//##### Second render pass - from eye's point of view//Calculate texture matrix for projectionMATRIX4X4 textureMatrix =biasMatrix * lightProjectionMatrix * lightViewMatrix;if (texTransform != NULL) cgGLSetParameter4fv(texTransform,textureMatrix);glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);glPushMatrix();makeCamera();makeLighting();makeGeometry();glPopMatrix();//##### End of second render pass - from eye's point of viewI still have trouble using/understanding that texture matrix. Why do I have to project it? I got the idea with those other matrices (MODELVIEW, PROJECTION), but I don't understand the texture matrix stuff. And without the knowledge of this I can't try out your advice to project an ordinary texture... :((btw: The texture matrix is given to the Cg shaders as shown above in the source code. No further modifications are made.)Thanks again!   ;Quote:Original post by NinjaSquirrelWell, I am not really sure if am missing something here. To test my shadow map generation, I glued the shadow map as an ordinary texture to a quad and it shows what you'd expect: a gray-scale image with the scene rendered from the light's point of view. With that successfully working, I suppose my texture coordinates for that quad are correct. But I have to admit that I am deeply troubled with 'projecting' a texture onto a scene (see below)...If you're setting up yourself the tcCoords, the only thing we can proof from here is that the shadow map has been correctly generated. That's very important however, you did well in testing this.(PS: I see you're using render-to-backbuffer-copy-to-texture... what are the performances? I originally used pbuffers).Quote:Original post by NinjaSquirrelThe shadow map generation is done like this: I position the camera at the light position, render the scene, store the aquired data in my DEPTH_COMPONENT-texture, switch back to my "real" camera position and render that scene again. My OpenGL-Code for that is (well, I actually have stolen most of this from a tutorial...):I see you're using LoadMatrix to set up light's MVP. I used a different approach but since we are sure the projection is fine (it's correctly showing the image after all) then this must not be an issue.Disabling smooth shading is pretty unlikely to give an interesting performance boost. I suggest to took it away (you'll disable color writes and color arrays anyway so why bother with it?).//##### Second render pass - from eye's point of view//Calculate texture matrix for projectionMATRIX4X4 textureMatrix =biasMatrix * lightProjectionMatrix * lightViewMatrix;if (texTransform != NULL) cgGLSetParameter4fv(texTransform,textureMatrix);glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);glPushMatrix();makeCamera();makeLighting();makeGeometry();glPopMatrix();//##### End of second render pass - from eye's point of viewI had some problems with the bias and I would be glad if you could give me more pointers on the solution you use here.Quote:Original post by NinjaSquirrelI still have trouble using/understanding that texture matrix. Why do I have to project it? I got the idea with those other matrices (MODELVIEW, PROJECTION), but I don't understand the texture matrix stuff. And without the knowledge of this I can't try out your advice to project an ordinary texture... :(Think about this: you're not projecting at all. You're taking points and transform by eye MVP. The same vertices pass thuru a tcCoord xform which is another MVP... you got two eye positions, one is relative to the eye, the other to the light.I believe it's easier with vertex programs than using the fixed functions pipe because you then have generic attributes and your mind is not messed up with tcCoords and such... you have attrib 0 and the other attribs.There's a line in your vertex shader wich does "OUT.SMTexCoord = mul(TexTransform,v);"which turns out to be just what you shall do. This should generate the coords correctly but I believe it's worth a try. If you project correctly the image shall be something likeLittle background on the image: this is viewed from light eye and shows the tcGen on a plane. Further tests used a more arbitrary geometry with the same algorithm (although the cage was later drawn with a different Vertex program).;Quote:Original post by KrohmIf you're setting up yourself the tcCoords, the only thing we can proof from here is that the shadow map has been correctly generated. That's very important however, you did well in testing this.(PS: I see you're using render-to-backbuffer-copy-to-texture... what are the performances? I originally used pbuffers).Performance ranges from "pretty bad" to "simply terrible"... :) I get something about 40fps. But I didn't give a single thaught on optimization yet, so there should be some speedup potential. I might try switching to pbuffers (another thing I am not fimiliar with ... yet! I'm still a newbie after all... :)Quote:Original post by KrohmDisabling smooth shading is pretty unlikely to give an interesting performance boost. I suggest to took it away (you'll disable color writes and color arrays anyway so why bother with it?).Yepp, point for you. This is actually part of that "stolen code" I found in a tutorial. Didn't pay much attention to the shading mode when I worked through the tutorial. I might remove this...Quote:Original post by KrohmI had some problems with the bias and I would be glad if you could give me more pointers on the solution you use here.I found a nice tutorial here on gamedev.net that focuses on rendering of realistic water surfaces. About half way down the page there is a section that takes a look at projective texturing ("A word on projective texturing"). I read that section after I had posted my last reply here and it really helped me understanding what is taking place when you're projecting a texture into a scene. They also set up that bias matrix there. Strange thing is, they use the transposed version of my bias matrix. (I tried out their version, didn't make any difference for my problem...) The URL is http://www.gamedev.net/reference/articles/article2138.asp.Anyway, after reading the gamedev-article and your reply, I understand why projective texturing is useful for shadow mapping and how it basically works. I'm still lost with my code, though. Even with that new knowledge I haven't been able to track down my mistake. I'll bet its some other nasty little bug that's hiding in a completely different part of my code... :)Thanks for your effort!   ;Quote:Original post by NinjaSquirrelPerformance ranges from "pretty bad" to "simply terrible"... :) I get something about 40fps. But I didn't give a single thaught on optimization yet, so there should be some speedup potential. I might try switching to pbuffersIf you're still reading this, don't waste time on pbuffers, go directly to "EXT_fbo".On my system, rendering a 300x300 window with a 128x128 pbuffer drives my 6600GT to 30-40fps. It's pretty slow, isn't it?I would have expected your performance to be higher but I really don't know what you're doing so this is not a surprise.Quote:Original post by NinjaSquirrelThe URL is http://www.gamedev.net/reference/articles/article2138.asp.Thank you very much for giving me a direct link! I lack connectivity and this would save me a ton of time searching!