   Per Pixel Lighting - Let's start over
  I've made probably a dozen posts asking questions trying to get my per pixel lighting done. It's been 5 days. I've read and programmed from sun-up to sun-down and I am getting nowhere. I don't think people understand my problems because the questions are on tid-bits in the middle of the shader. I made it more confusing by trying to throw normal mapping in, because I thought it was a requirement. So, let's start over and keep it nice and simple. I'll just start with the lighting equation, so we know what we need from the pixel shader. Then we can devise a way to get it.I'll edit this post and add more shader code as we work our way from back to front. That is, if I get answers (I hope and pray).------------------------------------------Goal: Perform per pixel lighting with up to 8 directional lights in one shader pass.------------------------------------------Here are some variables that the application will be passing to the shader, I'm just going to fill these out as we go://--------------------------------------------------------------------------------------// File: EffectPool.fx//// Contains shared effect variables and functions that remain the same every frame// for all effects.//--------------------------------------------------------------------------------------shared cbuffer Matrices{   matrix view  : View;   matrix projection : Projection;};shared cbuffer Lights{   struct AmbientLight   { float  intensity; float4 color;   }; struct DirectionalLight   { bool   enabled; float4 direction; float4 color;   }; AmbientLightambientLight;   DirectionalLight directionalLights[8];};[source lang="cpp]// Actual shader#include "EffectPool.fxh"matrix world : World;//-------------------// Diffuse Variables//// Diffuse Color can be a single color or sampled from a texturefloat4    diffuseColor    = float4(1.0f, 1.0f, 1.0f, 1.0f);Texture2D diffuseTexture;bool diffuseMapped   = false;   // true if texture is mapped to diffuse color//-------------------// Specular Variables//// Specular Color can be a single color or sampled from a texturefloat4    specularColor   = float4(1.0f, 1.0f, 1.0f, 1.0f);Texture2D specularTexture;bool specularMapped  = false;   // true if texture is mapped to specular colorfloatspecularExponent;-----------------------------------------------------------------------Here is the Blinn Phong function that I came up with//--------------------------------------------------------------------------------------// Blinn Phong//float3 Blinn_Phong(in float3 normal, in float3 viewer, in float3 light, in float2 texCoord){   float3 colorD   = diffuseColor.rgb;   float3 colorS   = specularColor.rgb; if( diffuseMapped )   {  colorD = diffuseTexture.Sample(smplLinear, texCoord).rgb;   } if( specularMapped )   {  colorS = specularTexture.Sample(smplLinear, texCoord).rgb;   } float3 half_vector = normalize(light + viewer);   float  HdotN  = max(0.0f, dot(half_vector, normal));   colorS   = colorS * pow(HdotN, specularExponent);   colorD   = colorD * max(0.0f, dot(normal, light)); return colorD + colorS;}We know that if there is more than one light, we can call this in a loop from the pixel shader. So, we need the input params for each light:1) a normal2) a view direction3) a light direction 4) a texCoord--------------------------------------------Question 1)My understanding is that, since this is per pixel lighting, all of these inputs need to be in respect to the pixel, not the vertex.From what I have read, the pixel shader interpolates its inputs from the vertex shader in order to make these inputs per pixel. Is that correct?Question 2)So how are we going to get these inputs to the lighting equation?float4 PS( PS_INPUT input ) : SV_Target{   float3 finalColor = {0,0,0};   // Calculate directional light contribution for(uint i = 0; i < 8; ++i)   { finalColor += BlinnPhong( ?, ?, ?, ?);   }   // We'll worry about alpha later   return float4(finalColor, 1.0f);}From what I've read, it is not enough to use the values from per vertex attributes. My belief is that these are incorrect when any deformation or scaling takes place. I am fuzzy, but I've read over and over that the first 3 inputs there should be obtained from a tangent space transformation.So let's decide what our PS inputs need to be and if we need to do anything with them to call the Blinn Phong equation in the loop.--------------------------------------------Let's stop there. I could add code from any of 25 attempts, but I want to keep it simple and understandable until we get each part resolved. I'll see what kind of answers I get and then edit this post.Thank you all for helping. I am completely lost even armed with my linear algebra book, 14 graphics books, and Google.[Edited by - brekehan on April 30, 2009 6:56:39 PM]
  you are making it more complicated than it needs to be. Obviously you can't do eight lights if you are using the vertex shader to calculate tangent light vectors.Instead, do something like this (read the Crytech paper on the crysis engine--they talk about it)...basically you do all the lighting calcs in the pixel shader. Its called world space lighting (not the same as using world space normal maps, still needs tangents).in the vertex shader:pass out tangent, normal, and possibly binormal (binormal can be calculated in the ps if you want)in the pixel shader:float3x3 worldSpace;worldSpace[0] = mul(In.tangent, matWorld);worldSpace[1] = mul(In.binormal, matWorld);worldSpace[2] = mul(In.normal, matWorld); float3 WorldNrm = normalize(mul(normalmap.rgb, worldSpace));you just transform the normal map itself into world space.Then you use the lighting vectors in world space, no need to transform, just make sure they are normalized.for each light{  light1=saturate(dot(lightvec,WorldNorm));etc....}now you can do as many lights as you want.   ;  To add on to Matt...1.) The rasterizer is responsible for interpolation of vertex attributes. Rasterization happens in between the vertex shader and pixel shader. The pixel shader is merely responsible for performing operations on the pixel. But the pixel shader will receive the interpolated values output from the vertex shader.2.) The view and light direction need to be uniforms in the shader. The values for these are passed in to the shader from the application.ex://shader valuesuniform float3 ViewDir;uniform float3 LightDir[8];//setting these values in the application (in D3D/c++)effect->SetValue("ViewDir", ViewDirection, sizeof(D3DXVECTOR3));effect->SetValue("LightDir[0]", LightDir0, sizeof(D3DXVECTOR3)); //sets the 0 lightDirectionThe per-pixel texture coordinates are the input to the pixel shader. So your PS_INPUT struct might look like:struct PS_INPUT{   //...   float2 TexCoord : TEXCOORD0;   //...};And you would simply pass the TexCoord to your lighting function.Since you transform your tangent space normal map to world space, there is no need to transform the light direction or view direction to tangent space.   ;Quote:Original post by Matt Aufderheideyou are making it more complicated than it needs to be. Obviously you can't do eight lights if you are using the vertex shader to calculate tangent light vectors....basically you do all the lighting calcs in the pixel shader. Its called world space lightingI figure as much.So what we need to decide (or I need to understand) is:1) What space do the light vectors need to be in if they to be calculated in the PS? and why (tangent space)?2) How are we going to get them in that spaceQuote:Original post by Matt Aufderheidein the vertex shader:pass out tangent, normal, and possibly binormal (binormal can be calculated in the ps if you want)in the pixel shader:float3x3 worldSpace;worldSpace[0] = mul(In.tangent, matWorld);worldSpace[1] = mul(In.binormal, matWorld);worldSpace[2] = mul(In.normal, matWorld); float3 WorldNrm = normalize(mul(normalmap.rgb, worldSpace));you just transform the normal map itself into world space.Then you use the lighting vectors in world space, no need to transform, just make sure they are normalized.for each light{  light1=saturate(dot(lightvec,WorldNorm));etc....}now you can do as many lights as you want.Ok, that is going over the entire approach so it introduces a slew of questions.I know you guys are smart and have done this a bunch of times. But for the unitiated like myself, it is just too much at once. I kind of wanted to just go step by step. Handling the whole thing at once is what I've been doing all week and it just confused the heck out of me.I'll just ask the biggest for now. Your approach requires a normal map. So, it begs the question...Is it indeed necessary to use a normal map?This is the way I started and it lead to a million posts and lots of frustration. It adds complexity for both the programmer (me) and the artist (me). Namely, the whole process of even getting a normal map and the difference between the coordinate space a 3rd party software uses compared to that of DirectX. I've poored over Maya and Max forums and can't even be sure I can get a normal map into my shader properly.Is there a way to get the light directions (transformed properly) in the PS without a normal map? Or is a normal map necessary in order to truly be "per pixel lighting"? If we absolutly need to, we can head down the normal map route again. Then I'll post a cube and a normal map to be used for testing. But I don't wanna if I don't have to. At least for now.[Edited by - brekehan on April 30, 2009 7:51:10 PM];  You don't need to use a normal map. If you don't care about deformation, then you can simply use the normals defined per-vertex. In that case you wouldn't need to construct the worldSpace matrix in Matt's post. You would simply take as input to the pixel shader the interpolated normal and use that for lighting.If you do care about deformation, you can either recalculate the normals when the model is deformed, or use a tangent space normal map.However, I say for now you simply get per-pixel lighting working without using normal maps. To do that, output the normal from the vertex shader, and have the pixel shader take a normal as input.So PS_INPUT could look like:struct PS_INPUT{   //...   float2 TexCoord : TEXCOORD0;   float3 Normal : TEXCOORD1;   //...};And pass this normal to your lighting function. Remember, you have to normalize the per-pixel normal as it can become un-normalized after interpolation.   ;Quote:Original post by glaekenYou don't need to use a normal map. If you don't care about deformation, then you can simply use the normals defined per-vertex. In that case you wouldn't need to construct the worldSpace matrix in Matt's post. You would simply take as input to the pixel shader the interpolated normal and use that for lighting.If you do care about deformation, you can either recalculate the normals when the model is deformed, or use a tangent space normal map.However, I say for now you simply get per-pixel lighting working without using normal maps. To do that, output the normal from the vertex shader, and have the pixel shader take a normal as input.So PS_INPUT could look like:struct PS_INPUT{   //...   float2 TexCoord : TEXCOORD0;   float3 Normal : TEXCOORD1;   //...};And pass this normal to your lighting function. Remember, you have to normalize the per-pixel normal as it can become un-normalized after interpolation.Ok, lets try that. Simple is better to start with. I can mess with normal maps after I get this approach working. Let me type some things up, try it out, and see if I get stuck again.I really appreciate the help. The 4th floor roof was really beginning to look tempting :) ;  Ok, I completed an attempt. It lights, but I do not trust it is lighing correctly.I set up a scene with 1 model at 0,0,0 1 directional light with a direction of 0,-1, 0camera at 0, 0, -250I moved the camera around a bit to see how the view vector changed the lighting.This is look into the +z axisThis is looking into the -z axisIf the light is shining from above, I don't see why the second image is has so much more light being reflected.So, when we left off, we decided to use the normals that are transformed in the VS. I was still wondering about how to transform the light and view vectors as I wrote the shader. I am certain I messed it up.Let's start with the Pixel shader//--------------------------------------------------------------------------------------// Pixel Shader//--------------------------------------------------------------------------------------float4 PS( PS_INPUT input ) : SV_Target{   // Calculate Ambient contribution   float3 colorA = Ambient(input.texCoord); // Calculate directional light contribution   float3 colorDS = {0,0,0}; for(uint i = 0; i < 8; ++i)   { // Add this light's contribution colorDS += Blinn_Phong(normalize(input.normal),normalize(input.view),   -directionalLights.direction.xyz,    input.texCoord);   } return float4(colorA.rgb + colorDS.rgb, 1.0);}You can see that I am using the directional light vector that is passed into the shader as a global variable. Since it is defined by the application in world space, I didn't think I needed to transform it all.The view direction is something that needs to be calculated. The view direction depends on which triangle or even which pixel we are looking at right? So, I figured I would calculate a view direction for each vertex in the VS and let the PS receive interpolated values (view direction at pixel).So, here is the Pixel Shader input structurestruct PS_INPUT{   float4   position  : SV_POSITION;   float2   texCoord  : TEXCOORD0;   float3   normal    : NORMAL;   float3   view : ViewDirection;  // vector from camera to pixel in world space};We are asking the VS for a projection space positiona non transformed texture coordinatea normal in world spacea view direction in world spaceTo provide those things, I wrote the following Vertex Shader//--------------------------------------------------------------------------------------// Vertex Shader//--------------------------------------------------------------------------------------PS_INPUT VS( VS_INPUT input ){   PS_INPUT output = (PS_INPUT)0; // Transform the incoming model-space position to projection space   matrix worldViewProjection = mul( mul(world, view), projection);   output.position  = mul( input.position, worldViewProjection); // Copy the texture coordinate   output.texCoord = input.texCoord; // Transform the incoming normal to world space without scaling   output.normal = mul(input.normal, (float3x3)worldInverseTranspose);   normalize(output.normal);   // Calculate the view direction vector in world space   float3 positionWorld  = mul( input.position, world).xyz;   float3 cameraPosition = view[3];   output.view = cameraPosition - positionWorld; return output;}So, here I Transform the position from object space to projection spaceCopy the texture coordinate, leaving it in object spaceTransform the normal to world space without any scaling.and wait.. This is one part I am unsure of. I use the world inverse transpose to get rid of scaling. I do it because I was told I needed to in case I ever have any non-uniform scaling. Is this part correct?I then calculate a view position by taking the camera's position and subtracting it from the vertices position in world space (without scaling)I am also unsure about this part.I got the camera's position from the 4th row of the view matrix. I believe that part is right.So here is the input structure for the Vertex Shaderstruct VS_INPUT{   float4 position  : POSITION;   float2 texCoord  : TEXCOORD;   float3 normal    : NORMAL;};and just in case I made an error, I did change a few float4s to float3s to do calculations on color without the alpha being considered. So for the sake of completeness, here is my lighting function//--------------------------------------------------------------------------------------// Calculates Ambient Color Component//float3 Ambient(float2 texCoord : TEXCOORD) : COLOR {   float3 colorA   = diffuseColor.rgb;   float3 colorE   = emissiveColor.rgb;   if(diffuseMapped)   {  colorA = diffuseTexture.Sample(smplLinear, texCoord).rgb;   }   if(emissiveMapped)   {  colorE = emissiveTexture.Sample(smplLinear, texCoord).rgb;   }   return colorA * (ambientLight.intensity * ambientLight.color.rgb) + colorE;}//--------------------------------------------------------------------------------------// Blinn Phong//float3 Blinn_Phong(in float3 normal, in float3 viewer, in float3 light, in float2 texCoord){   float3 colorD   = diffuseColor.rgb;   float3 colorS   = specularColor.rgb; if( diffuseMapped )   {  colorD = diffuseTexture.Sample(smplLinear, texCoord).rgb;   } if( specularMapped )   {  colorS = specularTexture.Sample(smplLinear, texCoord).rgb;   } float3 half_vector = normalize(light + viewer);   float  HdotN  = max(0.0f, dot(half_vector, normal));   colorS   = colorS * pow(HdotN, specularExponent);   colorD   = colorD * max(0.0f, dot(normal, light)); return colorD + colorS;}Now it is very possible that my normals being input to the shader are out of whack. Since this model came from a 3d modeler and there is always some y and z switching to do. So, I will try and get a cube to verify the normals are coming in right in the meanwhile. It will be awhile, because I have to write an exporter for max, since my old one was for maya. I recently switched to max 2010 from maya 6.0.1 to get up to date.EDIT: normalized the normal and view vectors that were coming into the PS, edited the source above to reflect the change. I was told to do it but forgot. However, the change didn't seem to make any effect on the result.I am also beginning to suspect the lighting equation. After running the application several times and moving the camera around from all points of view, I am noticing that the diffuse component seems strongest when I am looking down the y axis (as expected), but the specualr component is stongest when looking down the z axis (not expected). If it was simply 1 axis flipped somewhere, I'd suspect both components to be brightest from the same perspective.Still working on a exporting cube in max to see if my export process is to blame:( [Edited by - brekehan on May 1, 2009 12:05:50 AM];  I am still trying to figure out the problem here. I finished my 3dsmax exporter, or at least got it working. So, I made myself a unit cube, mapped a texture to the diffuse channel, set an ambient color of black.In the engine:I turned my ambient light intensity to 0Set my directional light color to 1 1 1 1Set the directional light direction to 0 -1 0For the geometry:I verified the output from the exporterVerified the input to the engine from the exported fileThe output from max matches the input to the engine.The input to the VS in PIX matches the input to the engine.So, there are no data problems. The problem has to be with the shader code.You can see more clearly what my problem is with these screenshotsIt is pretty clear that the diffuse light is behaving like it should, coming down from the positive Y axis to the origin and hitting the object. The texture shows up with the diffuse light.However, it is also clear that the specular light is NOT coming down the positive Y axis to the origin and hitting the object as it should. It is instead coming from somewhere close to the +Z axis. You can see the specular highlight there, and without the diffuse component near the bottom.So, there is something wrong with the specular component math in the shader code. I have no clue where it is going wrong!#include "EffectPool.fxh"matrix world  : World;matrix worldInverseTranspose : WorldInverseTranspose;//-------------------// Ambient Variables//// Ambient color is the same as diffuse color for this shader. It is rare they would beed to be different.// Emmisive color can be a single color or sampled from a texturefloat4    ambientColor    = float4(1.0f, 1.0f, 1.0f, 1.0f);Texture2D ambientTexture;bool ambientMapped   = false;   // true if texture is mapped to diffuse colorfloat4    emissiveColor   = float4(0.0f, 0.0f, 0.0f, 1.0f);    Texture2D emissiveTexture;bool emissiveMapped  = false;   // true if texture is mapped to emmisive color//-------------------// Diffuse Variables//// Diffuse Color can be a single color or sampled from a texturefloat4    diffuseColor    = float4(1.0f, 1.0f, 1.0f, 1.0f);Texture2D diffuseTexture;bool diffuseMapped   = false;   // true if texture is mapped to diffuse color//-------------------// Specular Variables//// Specular Color can be a single color or sampled from a texturefloat4    specularColor   = float4(1.0f, 1.0f, 1.0f, 1.0f);Texture2D specularTexture;bool specularMapped  = false;   // true if texture is mapped to specular colorfloatspecularExponent;//-------------------// Texture SamplersSamplerState smplLinear{   Filter   = MIN_MAG_MIP_LINEAR;   AddressU = Wrap;   AddressV = Wrap;};//--------------------------------------------------------------------------------------struct VS_INPUT{   float4 position  : POSITION;   float2 texCoord  : TEXCOORD;   float3 normal    : NORMAL;};struct PS_INPUT{   float4   position  : SV_POSITION;   float2   texCoord  : TEXCOORD0;   float3   normal    : NORMAL;   float3   view : ViewDirection;  // vector from camera to pixel in world space};//--------------------------------------------------------------------------------------// Lighting Models//--------------------------------------------------------------------------------------//--------------------------------------------------------------------------------------// Calculates Ambient Color Component//float3 Ambient(float2 texCoord : TEXCOORD) : COLOR {   float3 colorA   = diffuseColor.rgb;   float3 colorE   = emissiveColor.rgb;   if(ambientMapped)   {  colorA = ambientTexture.Sample(smplLinear, texCoord).rgb;   }   if(emissiveMapped)   {  colorE = emissiveTexture.Sample(smplLinear, texCoord).rgb;   }   return colorA * (ambientLight.intensity * ambientLight.color.rgb) + colorE;}//--------------------------------------------------------------------------------------// Blinn Phong//float3 Blinn_Phong(in float3 normal, in float3 viewer, in float3 light, in float2 texCoord){   float3 colorD   = diffuseColor.rgb;   float3 colorS   = specularColor.rgb; if( diffuseMapped )   {  colorD = diffuseTexture.Sample(smplLinear, texCoord).rgb;   } if( specularMapped )   {  colorS = specularTexture.Sample(smplLinear, texCoord).rgb;   } float3 half_vector = normalize(light + viewer);   float  HdotN  = max(0.0f, dot(half_vector, normal));   colorS   = colorS * pow(HdotN, specularExponent);   colorD   = colorD * max(0.0f, dot(normal, light)); return colorD + colorS;}//--------------------------------------------------------------------------------------// Vertex Shader//--------------------------------------------------------------------------------------PS_INPUT VS( VS_INPUT input ){   PS_INPUT output = (PS_INPUT)0; // Transform the incoming model-space position to projection space   matrix worldViewProjection = mul( mul(world, view), projection);   output.position  = mul( input.position, worldViewProjection); // Copy the texture coordinate   output.texCoord = input.texCoord; // Transform the incoming normal to world space without scaling   output.normal = mul(input.normal, (float3x3)worldInverseTranspose);   normalize(output.normal);   // Calculate the view direction vector in world space   float3 positionWorld  = mul( input.position, world).xyz;   float3 cameraPosition = view[3];   output.view = cameraPosition - positionWorld; return output;}//--------------------------------------------------------------------------------------// Pixel Shader//--------------------------------------------------------------------------------------float4 PS( PS_INPUT input ) : SV_Target{   // Calculate Ambient contribution   float3 colorA = Ambient(input.texCoord); // Calculate directional light contribution   float3 colorDS = {0,0,0}; for(uint i = 0; i < 8; ++i)   { // Add this light's contribution colorDS += Blinn_Phong(normalize(float4(input.normal, 1.0f).xyz),normalize(float4(input.view,1.0f).xyz),-directionalLights.direction.xyz,input.texCoord);   } return float4(colorA.rgb + colorDS.rgb, 1.0);}//--------------------------------------------------------------------------------------// Technique//--------------------------------------------------------------------------------------//--------------------------------------------------------------------------------------// Renders//technique10 RenderDefault{    pass P0    {   SetVertexShader( CompileShader( vs_4_0, VS() ) );   SetGeometryShader( NULL );   SetPixelShader( CompileShader( ps_4_0, PS() ) );    }}//--------------------------------------------------------------------------------------// File: EffectPool.fx//// Contains shared effect variables and functions that remain the same every frame// for all effects.//--------------------------------------------------------------------------------------shared cbuffer Matrices{   matrix view: View;   matrix projection    : Projection;};shared cbuffer Lights{   struct AmbientLight   { float  intensity; float4 color;   }; struct DirectionalLight   { bool   enabled; float4 direction; float4 color;   }; AmbientLightambientLight;   DirectionalLight directionalLights[8];};[Edited by - brekehan on May 5, 2009 9:50:07 AM];  I can't see the images in your latest post.   ;  Hmm, maybe the foward slashes instead of back slashes?I changed it. My server is up... can you see them now?