   A method to Gain the Z buffer, Device  independent
  I forget to write the subject in my last thread, faint, so I have to post it again here:It's nearly a mission imporsible to gain the data in Z-Buffer directly with Direct9. Normally, you can set a RenderTarget, write a Pixel Shader which outputs Z/W per pixel. Or you can depend on Z-Buffer Readable graphic cards, and copy the z-buffer surface to new surfaces. Both of these method are tough to be utilized in a real game engine.  I will explain why after I explain my method.My method is based on the fact that if Z buffer works, it should cull those pixels whichs depth are deeper than the corresponding pixels in Z-buffer. So I just test it by writing a bunch of surfaces with different depth to the back buffer with different colors, and acquire the result by render target.The steps are showed as follows:1. Complete drawing everything that you want the depth. For example: your terrain and your buildings.2. Replace the RenderTarget with a new surface which are used to read and gain the result later. It should be as small as it should be. I thought 256*128 is quit enough.3. Draw 256 surfaces in the post projection space with w = 0, z = 1 to 0, color = 256 to 0, namely far to near. Remember to shut lights and textures.4. Use the render target in your shader, such as a shadow map second pass. Cheer up, you can now get the Depth without an extra pass.There are tricks in step 3: You only need to build a Vertex Buffer once, then you can use it whenever you want to launch the "Depth Drawing"(Lets call this method like this for short). You must use the stencil buffer to accelerate this process. You can decide how the Z value increases through surfaces by what your scene looks like. Because in D3D projection matrix, 33 = F/(F-N), 43 = -FN/(F-N) (F=Far Clip, Near = Near Clip),  Depth in the post projection space should equal Zw = F(1 - N (1/Zv))/(F-N). You can figure  out this formular on your own, and you can find that Zw in Depth Space (post projection Space) is actually linear to 1/Zv (Zv is the Depth in View Space). As a result , you can set z like this : V1.z = (256.0 / (256.0 - 1.0)) * (1.0 - 1.0/(DOUBLE)(i+1));So the Z are linear to 1/i (i = 256 to 0). Make sure the Z is in [0, 1], because it is in post projection space of directx(OpenGl is [-1,1],depending on the projection matrix).You can find out in my codes that the surfaces I drawn are not parallel to the Near clip plate, because  I use the D3DPT_TRIANGLESTRIP in DrawPrimitive method in order to reduce bandwidth consumption. The surfaces are acctually looks lick overlapping Zs from the top. This trick has an extra benefit that the color values and the depths of pixels are interpolated automatically by the hardware.The result render target will be worked out by the graphic card like this : 1. The surfaces with z increasing are drawn from far to near. 2. Once a surface with less Z passes Z test, it also means it is the first surface pass the Z test. The depth (can gained by the color) of the surface is the closest one to the value in Z-buffer.3. it writes the stencil buffer to 1, telling the following pixel to skip.As you can see, there are many drawbacks in this method. First, it is highly pixel fill rate comsuming. It can be alleviated partially by making the render target smaller. Second, the precision is very low, because it can only provide 256 grads. You can try gain more grads with more faces, while it will surely consume more fill rate. This method is more suitable for Z correctiong, point z detection, and some other stages that don't require high z accuracy. Getting Z buffer is too tough a job in DX9. It can only be copy to a surface with same size and same format, but how can I use it anyway if they are the same. If you try to use a lockable depth surface, you are doomed. I try this with D16_Lockable and D32_Lockable, and once I lock, the fps drop from 600 to 12, even I unlock it immediatly without doing nothing and even the lock is called before BeginScene. I dont know whether other graphic cards will suffer this problem. The Lockable buffer is not pratical at all because even it can be locked, we have to do format convertion at lease once to gain the data in it, and it may be more slower than just render the scene twice with lights and textures shutted. In my opinion, why didn't the PS provide a register called iDepth, when there are already a oDepth avallable. It's baffling that once a segment get through all the tests and reach the Pixel Shader, it's depth has already been compared with the Z-buffer, and is it so tough to just pass the compare value to PS? Then we can have iDepth. If there is iDepth register(or Semantic) in PS, we can spare the gpu time to calculate z/w in the shadow map algorithm. Moreover, in the second pass of shadow map, we can also spare the gpu time to calcualte and pass the value of z/w in view space again.Newer DX versions have provided much easier way to get access to the depth buffer, however, it's not very pratical to ask out customer to buy a 8800 before buying our game, isn't it?Any people who has any thought about this method, feel free to reply. I am eager to chat about methods to get the Z buffer. codes are shown as follows:(I omit most trivial details)3.1 Build The Bufferstruct TypeRectVertex {  enum{fvf = D3DFVF_XYZRHW | D3DFVF_DIFFUSE,};  FLOAT x,y,z,w;  D3DCOLOR color;}enum{  numquos = numlevel,//256  numvertices = numquos * 2 +2,  vb_size = numvertices * sizeof(TypeRectVertex),};ZeroMemory(m_VBInMemory, sizeof(TypeRectVertex) * _countof(m_VBInMemory));_ASSERTE(pd3dDevice);_ASSERTE(numlevel %2 == 0 && numlevel <= 256);const D3DSURFACE_DESC* pSurfaceDesc = DXUTGetBackBufferSurfaceDesc();if(!pSurfaceDesc)  return;FLOAT fWidth = (FLOAT)pSurfaceDesc->Width;FLOAT fHeight = (FLOAT)pSurfaceDesc->Height;LPDIRECT3DVERTEXBUFFER9 lpVBTemp = NULL;HRESULT hr = pd3dDevice->CreateVertexBuffer(vb_size, 0, TypeRectVertex::fvf, D3DPOOL_MANAGED, &lpVBTemp, NULL);if(FAILED(hr))return;LPVOID lpData = NULL;if(SUCCEEDED(lpVBTemp->Lock(0, vb_size, &lpData, 0))){  {    TypeRectVertex * const pVertices = (TypeRectVertex*)m_VBInMemory;    _ASSERTE(pVertices);    for(int i = numvertices/2 - 1; i >= 0; i--){ TypeRectVertex V1; V1.x = i%2 == 0 ? 0 : fWidth;V1.y = 0;V1.w = 1;V1.z = (256.0 / (256.0 - 1.0)) * (1.0 - 1.0/(DOUBLE)(i+1));int nColor = (INT)(V1.z * 256.0f);V1.color = D3DCOLOR_XRGB(nColor, nColor, nColor);TRACE2("X:%f, Z:%f\n",V1.x, V1.z);TypeRectVertex V2;V2.x = V1.x;V2.y = fHeight;V2.w = 1;V2.z = V1.z;V2.color = V1.color;TRACE2("X:%f, Z:%f\n",V2.x, V2.z);int nPlace = numvertices - 2 - i * 2;pVertices[nPlace] = V1;pVertices[nPlace+1] = V2;   }   memcpy_s(lpData, vb_size, pVertices, vb_size);}  lpVBTemp->Unlock();  m_pVB = lpVBTemp;  lpVBTemp = NULL;}SAFE_RELEASE(lpVBTemp);3.2 State Setters: I use temporary struct which's constructor will record the states, and the destructor will apply the states. You should make sure these states are reverted to normal after the "Depth Drawing". You should understand these codes if you are familiar to those annoying D3DRS constants.ExtraStateApplying.m_pDevice = pd3dDevice;ExtraStateApplying.m_bZEnable = D3DZB_TRUE;ExtraStateApplying.m_bZWriteEnable = D3DZB_FALSE;ExtraStateApplying.m_Color = 0xffffffffff;ExtraStateApplying.tss_colorop = D3DTOP_SELECTARG1;ExtraStateApplying.tss_colorarg1 = D3DTA_DIFFUSE;ExtraStateApplying.dwCull = D3DCULL_NONE;StencilStateTemp.m_Enable = TRUE;StencilStateTemp.m_Fail = D3DSTENCILOP_KEEP;StencilStateTemp.m_ZFail = D3DSTENCILOP_ZERO;StencilStateTemp.m_Pass = D3DSTENCILOP_REPLACE;StencilStateTemp.m_Func = D3DCMP_GREATER;StencilStateTemp.m_Ref = 0x00000001;StencilStateTemp.m_Mask = 0xffffffff;StencilStateTemp.m_WriteMask = 0xffffffff;3.3 Drawpd3dDevice->DrawPrimitive(D3DPT_TRIANGLESTRIP, 0, numquos*2);   
  One way to get a depth texture with one pass is to simply render everything to a  floating point surface, and always output the depth to the alpha channel.. do alpha testing with clip...Doesnt work with transparent polygons however.I imagine you can fiddle with th numbers and pack it somewhere else also..   ;  I dont't think I am fiddling, man. I forgot to explain why I try to figure out such a tiresome method.  As you say, you can simply get the depth by render the scene. You mean render every thing again with a PS outputs Z/W or use mul-render target, right? What if there's no chance to render the scene again? That is the problem I am facing to. Believe me, I 've read lots of articles about shadows rendering. I am now working on a complex program, which codes have been accumulated to an suffering height. It is extremly diffical for me to insert codes in those render functions, because most of them are written months ago by others without serious planned and most of them are nearly refraction incompatible. The best way to implement codes to launch a clean Z-buffer rendering, as you memsion, is to rewrite many codes to insert "RenderShadow" method in those model classes and every scene object classes. Moreover, some of our trees do not have a shader. That's why I try to implement such a bizarrerie method, when the boss is urging a realtime shadow.I think few 3d programmers need to get z-buffer like this, if they have time to implement a depth rendering. I present this bizarrerie method just for exchanging ideas with you guys to figure out such adverse a situation. At lease, my method has one advantage, isn't it? It only needs z-buffer :>PS : who can tell me how to place an image on the thread?   ;Quote:Original post by AlenWeskerI dont't think I am fiddling, man. I forgot to explain why I try to figure out such a tiresome method.  As you say, you can simply get the depth by render the scene. You mean render every thing again with a PS outputs Z/W or use mul-render target, right? What if there's no chance to render the scene again? That is the problem I am facing to. Believe me, I 've read lots of articles about shadows rendering.I don't understand how your method solves the problem of not needing an extra pass. A normal shadow mapping algorithm works like this:1. Render scene from the viewpoint of the light2. Render the scene from the camera's viewpoint, comparing depth to the previous renderThis inherently draws the scene twice.If you're just talking about drawing step 1 twice, there's no need for this, since the scene itself need not be rendered. All you need is the Z buffer, and you can get that by rendering Z to a floating point texture.Which is why I say that I don't understand what the exact problem is.   ;Quote:Original post by AlenWeskerWhat if there's no chance to render the scene again? Actually my method means that you only render your main scene once...you just always render the depth to the alpha channel while rendering the colors...that's the point, as 'ET3D' says..I was just considering the idea that you wanted a copy of the depth buffer only from the camera position, not for shadow mapping. But for shadow mapping you need to render the scene from the light's point of view, so there is nothing in the z-buffer stored already, so your method makes no sense at all.   ;  Hey, I think few guys have met my situation, so it's not surprising that it is so hard to explain. Normally, Shadow Map really needs render the scene twice. If you have fully control of your shaders and your models, just do as what you guys say. And if only the shadow is needed ,just render the shadow at the same time in the second pass with PS. We don't even need to render the depth into the alpha in this situation.However, what if some objects are render with no shaders, and the codes have been  messed up into great chaos? The best way is to sort them and add codes to do as what you suggests. I will do that some day, but not now, for the sake that the refration of this kind of mess is not a one-day job. In order to just realize the shadow in a short time, I do like this:1. Just let those messed-up codes do what they are supposed to do, even some objects are rendered without a shader.2. Use the bizzare method I introduced above to get the depth.3. Render the moving objects only to gain a shadow map.4. Render the shadow using the depth map obtained in step 3,with every pixel being compared to the shadow map.It does not save a pass, and what I mean is that I don't need to modify any previous codes to get the depth-----no need to modify shaders, and no need to dig a hole into those codes in chaos.This bizzare method is temporary, however, it do provide a way to get the depth even you don't have shaders available, isn't it? I have no word to say if you insist I am fiddling.Actually, I do not need to render my shadow like this right now, because I just implement a simply way to have soft shadow volumes yesterday. Let the depth go to hell.   ;  Oh, I forgot one thing to be clarified--- some one has occupy the alpha channel to store data for post effects.   