   HLSL frac() Issues
  Yet another HLSL issue for me. Why are these things never simple?Anyway, it seems that whenever I use frac() something goes horribly wrong.I tried using it in my code and it wasn't giving the expected results, so I tried using it on my TexCoords to check I was doing it right and I got very strange results.Pos.xy has the range [0... 256], OneOverTexSize is 256.Normally I calculate my TexCoords as follows:Pos.xy * OneOverTexSizeWhich gives values in the range [0... 1] obviously.To test that frac() worked as expected I changed this to:frac(Pos.xy * OneOverTexSize)Surely this should also give values in the range [0... 1]?However, instead of sampling my texture correctly I just get a solid colour (as if it's sampling one point all over the texture).Am I misunderstanding what frac() does here?I'm using SM3.0 in case it matters.   
  Are you certain that Pos.xy is varying over the surface of the screen?  Try simply outputting the Pos.xy as a color to make sure.  You could also just try calling frac( float4(0.5,1.5,2.5,3.5) ) and you should get gray out everywhere.  I have used frac quite a few times, and it appears to work as advertised.If those things don't shed light on the topic, try running with the reference rasterizer to see if you get the same result.  If you are, then it is your program that is doing something incorrectly!   ;  I worked out the problem... and feel stupid now that I've realised what was wrong.Previously I was using multiple quads to cover my area [0... 256], but to reduce draw calls I've moved to just using 1. Which of course means that the only values I'm passing to my vertex shader are 0 and 256, which with frac() both come out as 0.I'll move the code using frac() into the pixel shader tonight and all should be fixed. I was trying to improve efficiency by keeping it in the vertex shader, but I'm not GPU bound yet and I'm sure I have better places to optomise.Just to check, the expected result of frac(-0.75) is 0.25? (As with -1.75, -2.75...?)[Edited by - adt7 on May 29, 2009 4:35:02 AM];  AFAIK, the result of frac(-0.75) should be 0.75.  I think it just removes the sign and integer portion.  I would try it out to make sure though...   ;  See, that is what I thought, but comparing float4(0.25, 0, 0, 1) and (float4(frac(-0.75), 0, 0, 1) as a pixel shader output they appear the same, whilst float4(0.75, 0, 0, 1) appears to be noticably brighter.I'll use the reference device to make sure it's not my hardware when I get time.Edit: I've just thought about this (I should do this more often) and frac(x) is defined as follows:frac(x) = x - floor(x)Therefore, for x = -0.75:floor(x) = -1frac(x) = -0.75 - (-1)frac(x) = 0.25Just thought I would post that for other people's reference.Thanks for the help Jason.   