   Per Pixel Lighting
  I calculate a matrix in my VS that is needed in my PS. How can I pass that matrix from the VS to the PS?[Edited by - brekehan on April 30, 2009 4:49:54 PM]
  Which shader model are you using?Is the matrix necessary on a per-vertex frequency?   ;Quote:Original post by DieterVWWhich shader model are you using?Is the matrix necessary on a per-vertex frequency?SM_4Yes, per vertex.;  So, it is easy to pass data across, though the meaning to the pixel shader is more restricted.For instances, you can pass your matrix out of the vertex shader like so:struct VS_OUT{float4 position : SV_POSITION;nointerpolation float4x4 yourMatrix : my_matrix;};Then, just have the pixel shader accept the same structure in.  Now, the real problem is that you probably don't want the matrix interpolated across the triangle for each pixel, so you'd use the nointerpolation modifier.  However, when doing so all of the pixels for the primitive will receive the matrix from the leading vertex of the triangle. This is probably not what you want depending on the scenario.Your best option depending on your needs is to do the calculation in the geometry shader, and then pass all 3 matrices (one for each vertex) to each pixel at the same time, also using nointerpolation.  That way each pixel would have a matrix for each vertex from the parent primitive.   ;  I am not sure if it needs to be interpolated or not.I am struggling through per pixel lighting, by rigging 20 non-working tutorials into one working shader :PIf I understand what I am doing correctly, I need the matrix to transform my light direction from world space to tangent space.In the tutorial it was originally transformed in the VS, because they had a single light. I was taking the approach of doing up to 8 directional lights by looping through them in the PS. So, I need the transform in the PS to transform each light's direction.I've never done a geometry shader and am really having trouble finding anything on how it works, or how to write one. I don't know how the inputs and outputs work for it. All I have gotten was that it is in between the VS and PS and can morph shape. If you have some links to some simple geometry shader examples it would be great.Here is my shader code so far. It is part of an effect pool, so I can pass view, projection, and lights one time for several effects://--------------------------------------------------------------------------------------// File: EffectPool.fx//// Contains shared effect variables and functions that remain the same every frame// for all effects.//--------------------------------------------------------------------------------------shared cbuffer Matrices{   matrix view: View;   matrix projection    : Projection;};shared cbuffer Lights{   struct AmbientLight   { float  intensity; float4 color;   }; struct DirectionalLight   { bool   enabled; float4 direction; float4 color;   }; AmbientLightambientLight;   DirectionalLight directionalLights[8];};Keep in mind, I am in the process of writing this. It isn't done. I am stuck with the world to tangent matrix being passed and my emissive component not working right. Right now, when I apply an emissive texture that only has the a few parts light, my entire model seems to almost light up to the point it is all white, even with an ambient intensity of .5 and emissive of 1. //------------------------------------------// General Lighting Equation//// The lighting equation (commonly used in real-time graphics) is combined from three // parts - ambient, diffuse and specular in this fashion://// I_total = I_ambient + I_diffuse + I_specular//// I_total    is the final color value// I_ambient  is the ambient color// I_diffuse  is the diffuse color// I_specular is the specular color for a given pixel////--------------------------------------------// Ambient component//// The ambient component simulates indirect light contribution (for example a light that// bounced off a wall and then reached the object) and emission of a light by the surface itself.// This type of light interaction is not computed in diffuse or specular part. This component// ensures that every pixel receives at least some minimum amount of color and will not remain// black during the rendering.//// I_ambient = (M_ambient * L_ambient) + M_emissive//// M_ambient is the ambient part of material properties (color or a texture value at current pixel)// L_ambient is the ambient light color of a scene// M_emissive specifies color of light that surface emits//// It can be clearly seen that groups of pixels lit only by ambient lighting component will// likely loose the effect of 3D (due to absence of shades), because the light contribution for this// part is constant. A cure for this is to ensure that every place in scene receives at least// minimum light amount from a light source (i.e. we can use some type of minor headlight// light). Another method is to pre-compute offline the ambient (and static diffuse) parts of scene// lighting using for example the radiosity method and then use results as textures.////------------------------------------------// Diffuse component//// The diffuse component is based on a law of physics called Lambert's law and has the// biggest base in the real world (and the interaction of photons and surfaces). Lambert's law// says that the reflected light intensity is determined by the cosine of the angle a between the// surface normal n and the light vector L (heading from a surface point to the light) for totally// matte (diffuse) surfaces. Both vectors are normalized. This lighting component is independent// of viewer’s position, because of the fact that for a perfectly diffuse surface the probability of a// new reflection direction is equal for every direction.// // The diffuse light intensity (I_diffuse) computation has the form:// I_diffuse = cos(a) = dot(n, L)// // We extend the equation to take into account// 1) The surface color M_diffuse at the current pixel,// 2) The light color - L_diffuse // 3) That the angle between the light and the normal has to be from 0 to PI/2//    because for greater angles, the normal is faced away from the light and is not being lit.//// I_diffuse = max( dot(n, L), 0) * M_diffuse * L_diffuse////------------------------------------------// Specular component//// We use the specular part of lighting equation to simulate shiny surfaces with the// highlights. A very popular model is called the Phong reflection model and it computes// light intensity as the cosine of the angle between the normalized light vector, reflected around// the point normal, and the normalized vector from the point to the position of the viewer,// powered by the shininess value that specifies an amount of reflection. This equation has little to// do with the real world illumination but it looks good under most conditions and is easy to// compute.//// Phong's specular equation then is:// r = 2 * (dot(n, L) * n - L// I_specular = cos(a)^shininess = (dot(r, v)^shininess//// r is the reflection vector// n is the face normal// L is the normalized light vector// v is the vector from the point to the viewer// shininess is an amount of reflection//// If a dot product of n and v is less than zero (angle is greater than PI/2), then a pixel is// facing away from the light and lighting should not be computed.// // We extend the equation to take into account// 1) The material's specular intensity modifier M_specular (in real-time graphics commonly called the gloss map)// 2) The light's color L_specular// // r = 2 * (dot(n, L) * n - L// I_specular = (dot(r, v)^shininess * M_specular * L_specular#include "EffectPool.fxh"matrix world : World;//-------------------// Ambient Variables//// Ambient color is the same as diffuse color for this shader. It is rare they would beed to be different.// Emmisive color can be a single color or sampled from a texturefloat4    emissiveColor   = float4(0.0f, 0.0f, 0.0f, 1.0f);    Texture2D emissiveTexture;bool emissiveMapped  = false;   // true if texture is mapped to emmisive color//-------------------// Diffuse Variables//// Diffuse Color can be a single color or sampled from a texturefloat4    diffuseColor    = float4(1.0f, 1.0f, 1.0f, 1.0f);Texture2D diffuseTexture;bool diffuseMapped   = false;   // true if texture is mapped to diffuse color//-------------------// Specular Variables//// Specular Color can be a single color or sampled from a texturefloat4    specularColor   = float4(1.0f, 1.0f, 1.0f, 1.0f);Texture2D specularTexture;bool specularMapped  = false;   // true if texture is mapped to specular colorfloatspecularExponent;//-------------------// Texture SamplersSamplerState smplLinear{   Filter   = MIN_MAG_MIP_LINEAR;   AddressU = Wrap;   AddressV = Wrap;};//--------------------------------------------------------------------------------------struct VS_INPUT{   float4 position  : POSITION;   float2 texCoord  : TEXCOORD;   float3 tangent   : TANGENT;   float3 biTangent : BITANGENT;   float3 normal  : NORMAL;};struct PS_INPUT{   float4 position  : SV_POSITION;   float2 texCoord  : TEXCOORD0;   float3 tangent   : TEXCOORD1;   float3 biTangent : TEXCOORD2;   float3 normal  : TEXCOORD3;   float3 viewDir   : TEXCOORD4;   };//--------------------------------------------------------------------------------------// Calculates Ambient Color Component//float4 Ambient(float2 texCoord : TEXCOORD) : COLOR {   float4 colorA   = diffuseColor;   float4 colorE   = emissiveColor;   if(diffuseMapped)   {  colorA = diffuseTexture.Sample(smplLinear, texCoord);   }   if(emissiveMapped)   {  colorE = emissiveTexture.Sample(smplLinear, texCoord);   }   return colorA * (ambientLight.intensity * ambientLight.color) + colorE;}//--------------------------------------------------------------------------------------// Vertex Shader//--------------------------------------------------------------------------------------PS_INPUT VS( VS_INPUT input ){   PS_INPUT output = (PS_INPUT)0; // Transform the incoming model-space position to projection space   matrix worldViewProjection = mul( mul(world, view), projection);   output.position  = mul( input.position, worldViewProjection);   // The world space vertex position is useful for later calculations   float3 positionWorld = mul( input.position, world).xyz;    // The per-vertex attributes are in model space, so transform them to world space   // before generating a world->tangent space transformation   float3 vTangentWS   = mul( input.tangent, (float3x3)world);   float3 vBiTangentWS = mul( input.biTangent, (float3x3) world);   float3 vNormalWS    = mul( input.normal, (float3x3) world); // Generate a world space to tangent space transformation   float3x3 worldToTangentSpace;   worldToTangentSpace[0] = normalize( vTangentWS );   worldToTangentSpace[1] = normalize( vBiTangentWS );   worldToTangentSpace[2] = normalize( vNormalWS ); output.tangent   = worldToTangentSpace[0];   output.biTangent = worldToTangentSpace[1];   output.normal    = worldToTangentSpace[2];   output.viewDir   = float3(view[3][0], view[3][1], view[3][2]) - positionWorld;   output.viewDir   = mul( worldToTangentSpace, output.viewDir); output.texCoord = input.texCoord; return output;}//--------------------------------------------------------------------------------------// Pixel Shader//--------------------------------------------------------------------------------------float4 PS( PS_INPUT input ) : SV_Target{   float4 finalColor = Ambient(input.texCoord); return finalColor;}//--------------------------------------------------------------------------------------// Technique//--------------------------------------------------------------------------------------//--------------------------------------------------------------------------------------// Renders//technique10 RenderDefault{    pass P0    {   SetVertexShader( CompileShader( vs_4_0, VS() ) );   SetGeometryShader( NULL );   SetPixelShader( CompileShader( ps_4_0, PS() ) );    }};  Since you are doing per pixel lighting (and I saw your other post about normal maps) you should in such a case only need 1 tangent frame per triangle.  There are actually 2 options here, if you are normal mapping an object that has a fixed orientation with respect to the texture, then you can actually use the normals from the normal map verbatim provided that they are authored together.  You'll have to transform the normals the same way that the vertices were transformed, and this won't work if your object is skinned and can deform since each vertex is transformed differently across the triangle.  This is object space normal mapping.Otherwise, if you have a normal map that can be applied to an object with any possible starting orientation, then you will need a tangent frame so that you know how to orient the normals from the texture look up.   In this case you should only need one tangent frame per face.  With this in mind, you could calculate the tangent frame in the geometry shader, and then pass it out to the pixel shader.  Each pixel for the face would get the same matrix and can be used to orient the normals correctly.  Of course, you have to keep in mind when to apply the world/view/projection matrix to your vertices if your doing this calculation on the fly.  I suggest the geometry shader since you'll only need the frame on a per-face frequency.  You'd use per-vertex frequency if you were doing per-vertex lighting.  This is tangent space normal mapping.struct GS_IN{float4 position SV_POSITION;flaot2 texcoord : TEXCOORD;};struct GS_OUT{  float4 position : SV_POSITION;  float2 texcoord : TEXCOORD;  float3x3 tFrame : TangentFrame;};[maxvertexcount(3)]void main( triangle GS_IN vertex[3]; inout TriangleStream<GS_OUT> stream ) {// calculate tangent, bitangent, normal here// now write the three vertices to the stream.for( uint i = 0; i < 3; ++i ){GS_OUT gso;// pack data into GS_OUT structure for the current vertex here.stream.Append( gso );}stream.RestartStrip();}In the pixel shader you'll have a PS_IN struct with the float3x3  TangentFrame coming in, which you'll use to in the transformation the normals from the normal map.[Edited by - DieterVW on April 30, 2009 5:36:49 PM];  Thanks, I will make an attempt at a geometry shader in the morning. I'll go back to PIX debugging a cube and try to verify it too.   ;  I have some spots in the calculation of the tangent frame that I am not sure about now.Also, do I still need this tangent frame if I am not doing normal mapping, but to get the viewDir and lightDir per pixel in the PS?Here is my code so far with questions in comments://--------------------------------------------------------------------------------------// Geometry Shader//--------------------------------------------------------------------------------------[maxvertexcount(3)]void main(triangle GS_INPUT input[3] : SV_POSITION, inout TriangleStream<PS_INPUT> stream ) {   // Calculate tangent frame   float2x3 q;   q[0] = input[1].position - input[0].position;   q[1] = input[2].position - input[0].position;   float2x2 s1;   s1[0] = float2(input[1].texCoord.x - input[0].texCoord.x, input[1].texCoord.y - input[0].texCoord.y);   s1[1] = float2(input[2].texCoord.x - input[0].texCoord.x, input[2].texCoord.y - input[0].texCoord.y); float2x2 s2;   s2[0][0] =   s1[1][1]; s2[0][1] = -(s1[0][1]);   s2[1][0] = -(s1[1][0]);   s2[1][1] =   s1[0][0]; float    rdet   = 1.0f / determinant(s1);   float2x2 sinv = mul( rdet, s2); float2x3 tb;   tb = mul( sinv, q); // What about the normal if I didn't supply a normal map?   // Should I just use the normal that comes in with the vertex data as row 3 of the tangent frame?    // This part is unexplained. I can't visualize what's going on here   // Shouldn't we "orthogonalize" all of the vectors ( tangent and bi-tangent )?   // What is the handedness bit doing?   //   // Gram-Schmidt orthogonalize   // tangent[a] = (t - n * Dot(n, t)).Normalize();   // // Calculate handedness   // tangent[a].w = (Dot(Cross(n, t), tan2[a]) < 0.0F) ? -1.0F : 1.0F;    float3x3 tangentFrame;   tangentFrame[0] = tb[0];   tangentFrame[1] = tb[1];    // Now write the vertices to the stream   for(uint i = 0; i < 3; ++i)   { PS_INPUT output; // Transform the view direction into tangent space // Do this here or in the PS? output.viewDir   = float3(view[3][0], view[3][1], view[3][2]) - input.positionW; output.viewDir   = mul( worldToTangentSpace, output.viewDir);  output.texCoord = input.texCoord;  stream.Append(output);   }   stream.RestartStrip();};  First, I had a small error in my gs example, for stream inputs, remove the SV_POSITION semantic from the end of the vertices[3] declaration.  You can see the edited version of my post.You have 3 vertices in your gs, so you can use these to calculate the normal of the polygon with a simple cross product of the two sides of the triangle.If you are not doing tangent space normal mapping, then you won't need a tangent frame.  Perhaps it's easiest to get everything working and then come back to this.Object space normal mapping is easier.  Make sure that all of your object's polygons have different UV's.  Then, write a shader in your favorite DCC app (max,maya,etc.)  that just looks up the current normal of the underlying object and then bake out a texture for that object.  You might be able to find people who have posted shaders for doing this if you don't know how. Then, when doing the normal texture lookup, transform the normal with the rotational part of the full transformation matrix that was applied to the vertices of the model. (world and view matrix)The lights will have their own local-to-world matrix which will be used on the light direction vector.  Then use the same view matrix  (world-to-camera) to get the light vector into the same space as your model/normal are for shadering purposes.  (again, just the rotation part of the matrix is needed for transforming these vectors)   ;  I think I am going to start over. I have snippets all over the place and can't get a thing working. I'd like to take a top down approach, throw out the normal mapping, and make everything as simple as possible.When I get that working, perhaps I can investigate the normal mapping.Right now I can't even light things at all.Here is a fresh start in an attempt at consolidating all my various questions over the last week on this topic.