   World space position from linear depth
  I'm trying to get the world space position of a pixel from a buffer that stores the view-space depth of each pixel.I've been reading this: http://mynameismjp.wordpress.com/2009/05/05/reconstructing-position-from-depth-continued/and I don't understand a couple of things:First, how do you calculate the view space position of the frustum corners? My first thought was to simply multiply the four corners of a full screen quad in clip space by an inverse view-projection matrix. Would this work?Secondly, I don't get how you then get a ray from this data. So you have the view space position of the corners, where do you get a ray the goes from the corner to the far plane? Wouldn't the direction of said ray always be (0, 0, 1) in view space?Help much appreciated!   
  If you've got the positions of the corners, then a ray to a corner is cornerPos - cameraPos.If cameraPos is the origin/zero (because it's view-space), then the ray is just cornerPos.When you draw your fullscreen quad, you pass the corner positions in as vertex attributes, which are output by the vertex shader, interpolated, and then input to the pixel shader. So, by the time the pixel shader gets a hold of cornerPos, it's actually a weighted average (blend) of the original 4 cornerPos values, giving you a ray in the right direction.   ;  Ah OK, when corners where mentioned I was thinking of the near plane corners of the frustum.Now, how do I get the corner positions?   ;Suppose:float fx = camera's field of view in the x directionfloat fy = camera's field of view in the y directionfloat fp = distance of the far clip planeThen the four corners of your screen aligned quad in view space are:Corner.TopLeft= (-tan(fx), 1,  tan(fy)) * fpCorner.TopRight    = ( tan(fx), 1,  tan(fy)) * fpCorner.BottomLeft  = (-tan(fx), 1, -tan(fy)) * fpCorner.BottomRight = ( tan(fx), 1, -tan(fy)) * fp;  You don't want to take the quad positions in clip-space and transform them by the inverse view*projection. If you want a view-space position it's fine to do that and transform by the inverse projection, but if you want a world-space position what you actually want is the view-space corner positions rotated by the camera's world matrix. So something like this would work:for(int i = 0; i < 4; +i){    float3 cornerVS = mul(float4(cornerCS, 1.0f), invProjection);    float3 cornerWS = mul(cornerVS, cameraWorld);};  OK, so now I've constructed the corner positions as posted above and I'm creating the ray like this:Vertex shader:output.FrustumPositionWS = mul(input.FrustumPositionVS, CameraInvView);Pixel shader:float pixelDepth = tex2D(DepthMapSampler, input.TexCoords);float3 WorldPosition = CameraPosWS + ((1 - pixelDepth) * input.FrustumPositionWS); And I'm getting strange results. If I output the position as colour, the colours seems to move and change as the camera moves which obviously isn't correct.In-between a transition between colours here:A different camera angle (The top half contains no geometry so you can see a different, odd pattern):;  I've changed it slighty, the vertex shader now reads:float4 csPos = output.Position;csPos.z = 0.0f;csPos.w = 1.0f;float4 FrustumPositionVS = mul(csPos, InvProjection);FrustumPositionVS /= FrustumPositionVS.w;FrustumPositionVS.z = (FrustumPositionVS.z + 1000.0f ) * -1;FrustumPositionVS.w = 1.0f;output.FrustumPositionWS = mul(FrustumPositionVS, CameraInvView);where csPos is the clip-space position of that vertex. For some reason the output FrustumPositionWS remains almost exactly the same for each vertex. In fact for the two top vertices it is the same. Clearly I'm missing something here. The colours still change with camera movement too.   ;Okay, I think something needs to be clarified in light of the other posts.  When you multiply a vertex position by the projection matrix and hand it off to the GPU some additional processing happens behind the scenes to calculate pixel coordinates and depth values. Inverting the projection matrix and using the result to transform a screen-space position (PixelX, PixelY, Depth) gives you back nonsense; It doesn't matter how you scale (PixelX, PixelY) or if the depth value has been made linear or not.  Alas, it is possible to get a view-space, world-space or even object-space position from (PixelX, PixelY, Depth).  I've done it many times and will try to explain...   ;  Any explanation is useful! I'm finding all of this very interesting and I'm sure getting to know this sort of stuff a little deeper will be useful!   ;Why do I get to type this in a font that's aligned but it's not displayed that way?  This is going to be hard to read...this is our projection matrix:P = | Sx 0  0   0 |    | 0  Sy 0   0 |    | 0  0  Sz Tz |    | 0  0  1   0 |putting the point (x, y, z) in projection space:| Sx 0  0   0 | | x | | Sx * x || 0  Sy 0   0 |*| y |=| Sy * y || 0  0  Sz Tz | | z | | Sz * z + Tz || 0  0  1   0 | | 1 | | z |and to get to the screen space point (px, py, dz) we divide the result by w like so:| px | |(Sx * x) / z|   | py |=|(Sy * y) / z| | dz | |(Sz * z + Tz) / z|   | 1  | | 1|  At this point dz gets put in the depth buffer and (px, py) are in what is usually called normalized device coordinates (NDC-space).  In NDC-space [-1, 1] is the top left corner of the screen and [1, -1] is the bottom right corner of the screen.  This is an important detail.HLSL code to go from screen-space to view-space:float3 ScreenSpaceToViewSpace(float px, float py, float dz){    float3 v;    v.z  = g_vProjectK.w / (dz - g_vProjectK.z)    v.xy = (float2(px, py) * v.zz) / g_vProjectK.xy}Where g_vProjectK is a float4 you put in some constant register and is equal to: (Sx, Sy, Sz, Tz) from the projection matrix at the top.The derivation isn't difficult but I'll skip it for now unless someone wants to see it.The thing about this function is that dz is straight from the depth buffer, it has not been made linear.  Additionally (px, py) is in NDC-space, not texture space or pixel space.  Generally in the pixel shader you have a texture coordinate or the actual pixel coordinates so this must be taken into acount.  Finally, the result is in view-space which means it must be transformed by some matrix to get it in the space you want.Now, you are probably wondering: but what is all this nonsense about world-space vectors pointing to the fustrum corners and linear depth values?  The fustrum-corner-trick is an optimization that is significantly faster than the straight forward approach I just explained.  Why?  Well, that little function isn't the sort of thing GPUs can vectorize and after the function runs you are probably looking at a 4x4 matrix transform.Next post will be about getting the fustrum-corner-trick up and running.