   [SOLVED] Vertex shader newbeness with meshes
  Hello!I have a VERY basic vertex shader.It is supposed to do some diffuse lighting but as it wasn't working,I decided to just make it output everything in red.Here's the code (the last lines colors in red):matrix ViewMatrix;matrix ViewProjMatrix;vector AmbientMtrl;vector DiffuseMtrl;vector LightDirection;vector DiffuseLightIntensity = {0.0f, 0.0f, 1.0f, 1.0f};vector AmbientLightIntensity = {0.0f, 0.0f, 0.2f, 1.0f};struct VS_INPUT{    vector position : POSITION;    vector normal   : NORMAL;};struct VS_OUTPUT{    vector position : POSITION;    vector diffuse  : COLOR;};VS_OUTPUT Main(VS_INPUT input){    VS_OUTPUT output = (VS_OUTPUT)0;    output.position = mul(input.position, ViewProjMatrix);    LightDirection.w = 0.0f;    input.normal.w   = 0.0f;    LightDirection   = mul(LightDirection, ViewMatrix);    input.normal= mul(input.normal,   ViewMatrix);    float s = dot(LightDirection, input.normal);    if( s < 0.0f )   s = 0.0f;    output.diffuse = (AmbientMtrl * AmbientLightIntensity) + (s * (DiffuseLightIntensity * DiffuseMtrl));    output.diffuse.r=1.0f; //This was added as a test    output.diffuse.g=0.0f; //This was added as a test    output.diffuse.b=0.0f; //This was added as a test   return output;}The vertex shader compiles fine, I set all the shader constants and get no problem. My first person camera still works.BUTthe ID3DXMesh I'm rendering is rendered plain black except for the eyes which are normally shaded - plain red.Ok so it's supposed to be a robot, which's mesh is divided in 4 subsets.look what it gives:Can someone explain why is my vertex shader only coloring my robot's eyes?EDIT: It might be important for me to add that my mesh is textured...[Edited by - Trillian on December 9, 2005 1:00:25 PM]
  Someone correct me if i'm wrong, but I thought you needed a pixel shader to do the colouring? The vertex shader just transforms the vertices, then passes the fragment on to a pixel shader to be coloured. Or can you just do vertex colouring only? If you can, then isn't the input.normal.w   = 0.0f; line wrong, don't you need a 1 there?   ;  The "LightDirection" variable should contain a vector from the vertex to the light source.  If you're using a directional light, then you can set this value from the main application.  If you're using a positional light, you need to set the position as a constant to your vertex shader and compute the light direction inside the shader itself.As for your model rendering black with red eyes, are you using the fixed function pipeline to render part of the model?  If your vertex shader is running and the output is just a constant red color, your entire mesh should be red.  Perhaps there is some kind of mistake in the calls that render the mesh?@theZapper, it is possible to use vertex or pixel shaders individually.  You can use vertex shaders and compute lighting per vertex without needing to use pixel shaders.  If you want to compute per-pixel lighting, then you'll need to use a pixel shader.neneboricua   ;Quote:Original post by theZapperisn't the input.normal.w   = 0.0f; line wrong, don't you need a 1 there?No it's not wrong, I just checked it. and thanks neneboricua, I wasn't sure about that myself.   ;  Oh I think I understand the problem here!In my mesh render code, the code changes the device's material and textures!Could that deactivate the shader?   ;Quote:Original post by TrillianIn my mesh render code, the code changes the device's material and textures!Could that deactivate the shader?It could be.  Remember that calls like IDirect3DDevice9::SetMaterial and IDirect3DDevice9::SetTexture are meant for the fixed funciton pipeline.If you're using shaders through the ID3DXEffect interface, you need to use ID3DXEffect::SetValue and ID3DXEffect::SetTexture instead.  If you're using shaders but without the ID3DXEffect interface, you need to set these values using the ID3DXConstantTable interface.If it doesn't work, try posting the part of your mesh render code that deals with this.  Remember to use [ source ] and [ / source ] tags on the code.neneboricua   