   Pick Ray Problem
  Hey all,I wrote this routine, the function of which is to return a point in 3D space based on a mouse click. A simple pick-ray routine. In this routine, the x and y point is the point on the screen clicked, the z is a value from 0 to 1 which represents the distance along the view frustum to select the point's plane (0.0 would be on the near z plane, 1.0 would be on the far z plane).Here's the code:D3DXVECTOR3 LevelEditorIndoorClass::GetClosestGridPoint(int CameraNumber, float x, float y, float z)    {    D3DXVECTOR3 Pick;    D3DXMATRIX ProjectionMatrix;    D3DXMATRIX ViewMatrix;    D3DXMATRIX InvViewMatrix;    D3DXVECTOR3 PickRayDir;    D3DXVECTOR3 PickRayOrig;    //calculate the projection matrix    if (CameraType[CameraNumber] == CAMERA_3D)   D3DXMatrixPerspectiveFovLH(&ProjectionMatrix, D3DX_PI/4.0f,    ViewPort[CameraNumber].Width/ViewPort[CameraNumber].Height,    1.0f, 1000.0f);    else   D3DXMatrixOrthoLH(&ProjectionMatrix,    ViewPort[CameraNumber].Width * Camera[CameraNumber].GetCameraZoom(),   ViewPort[CameraNumber].Height * Camera[CameraNumber].GetCameraZoom(),   1.0f, 1000.0f);    //calculate a pick ray (screen space)    x -= ViewPort[CameraNumber].X;    y -= ViewPort[CameraNumber].Y;    Pick.x =  (((2.0f * x) / ViewPort[CameraNumber].Width) - 1) /    ProjectionMatrix._11;    Pick.y = -(((2.0f * y) / ViewPort[CameraNumber].Height) - 1) /ProjectionMatrix._22;    Pick.z =  1.0f;    //calculate the inverse view matrix    ViewMatrix = Camera[CameraNumber].GetViewMatrix();    D3DXMatrixInverse(&InvViewMatrix, NULL, &ViewMatrix);    // Transform the screen space pick ray into 3D space    PickRayDir.x  = Pick.x*InvViewMatrix._11 + Pick.y*InvViewMatrix._21 +Pick.z*InvViewMatrix._31;    PickRayDir.y  = Pick.x*InvViewMatrix._12 + Pick.y*InvViewMatrix._22 + Pick.z*InvViewMatrix._32;    PickRayDir.z  = Pick.x*InvViewMatrix._13 + Pick.y*InvViewMatrix._23 + Pick.z*InvViewMatrix._33;    D3DXVec3Normalize(&PickRayDir,&PickRayDir);    PickRayOrig.x = InvViewMatrix._41;    PickRayOrig.y = InvViewMatrix._42;    PickRayOrig.z = InvViewMatrix._43;    z = z*(1000.0f - 1.0f) + 1.0f;    return PickRayOrig + PickRayDir * z;    }My interface has four cameras on it (most of which are orthogonal, but one which is perspective). In order to test this routine, I called it twice and got two points (one on the near z plane, one on the far z plane):TempObject.StartPoint = GetClosestGridPoint(ActiveCamera, x, y, 0.0f);TempObject.EndPoint = GetClosestGridPoint(ActiveCamera, x, y, 1.0f);I then drew a line using these two points. I realize that (if this was working correctly) for the camera where the user clicks, you would only get a point, but the other three cameras (which show the same scene from different views) would give a line segment.However, it's not working as I would like. What I'm actually getting is the near point being right, but the far point being utterly wrong.Does anyone see anything wrong with my math or algorithm? Because I'm just not seeing it :)Thanks, Ron   
  I can't see anything obviously wrong with what you're doing... however, you might find it useful to look into D3DXVec3Unproject() - it does pretty much the same thing as you've tried to implement. The reason I suggest it is that you can use it as a reference - make sure you've got the right parameters and you're using the correct results etc...hthJack   ;  Interesting! I didn't know about that function, thanks for pointing it out. It takes the vector into object space, which I don't want, but if I feed it an identity matrix for the pWorld parameter, I expect it would get me into world space, yes? If it does that, I could replace most of my routine with that one line, heh. Sometimes you just gotta love D3DX :);  As far as I'm aware, passing in the identity matrix should effectively skip the object-space part as you require it to.D3DX is a pretty neat library [smile]Jack   ;  Thanks again Jack for the advice, D3DXVec3Unproject() worked perfectly. I still don't have any clue why my routine didn't work, but I'll get over it :)Final routine, in case anyone ever needs something similar:/---------------------------------------------------------------------------D3DXVECTOR3 LevelEditorIndoorClass::GetClosestGridPoint(int CameraNumber, float x, float y, float z)    {    D3DXVECTOR3 Pick;    D3DXMATRIX ProjectionMatrix;    D3DXMATRIX ViewMatrix;    D3DXMATRIX WorldMatrix;    D3DXVECTOR3 Result;    //calculate the projection matrix    if (CameraType[CameraNumber] == CAMERA_3D)   D3DXMatrixPerspectiveFovLH(&ProjectionMatrix, D3DX_PI/4.0f,    ViewPort[CameraNumber].Width/ViewPort[CameraNumber].Height, 1.0f,   1000.0f);    else   D3DXMatrixOrthoLH(&ProjectionMatrix,    ViewPort[CameraNumber].Width * Camera[CameraNumber].GetCameraZoom(),   ViewPort[CameraNumber].Height * Camera[CameraNumber].GetCameraZoom(),    1.0f, 1000.0f);    //get the view matrix    ViewMatrix = Camera[CameraNumber].GetViewMatrix();    //put an identity matrix into the world matrix//(we don't want object space, we want world space)    D3DXMatrixIdentity(&WorldMatrix);    //the point to find    Pick.x = x;    Pick.y = y;    Pick.z = z;    D3DXVec3Unproject(&Result, &Pick, &ViewPort[CameraNumber],&ProjectionMatrix, &ViewMatrix, &WorldMatrix);    return Result;    } [Edited by - RonHiler on December 5, 2005 10:47:04 AM];  Hi again,I'm just looking for a clarification on something (just for my own knowledge), it has to do with these lines of the code:Pick.x =  (((2.0f * x) / ViewPort[CameraNumber].Width) - 1) / ProjectionMatrix._11;Pick.y = -(((2.0f * y) / ViewPort[CameraNumber].Height) - 1) / ProjectionMatrix._22;What I have here is the width and height of my viewport. I have five viewports on my screen (one that covers the entire screen, and four "sub" views). I presumed that I should use the viewport dimensions that was the "active camera", which covers roughly 1/4 of the visible screen.However, on further reflection, I'm not sure now if that's correct. Should I instead be using the viewport that corresponds to the full screen (since x and y are in fullscreen coords)? That does seem to make a certain amount of sense, and I'm wondering if perhaps that was my mistake.I read the docs and anything I could find on google, but they universally presume you are using a single viewport, and don't say anything about if you have sub views.Thanks for any input. I'm just asking for my own edification (since Jack's advice did solve my problem), I just hate stuff that I don't understand :)Ron   