   Picking - Point on near plane - Why?
  Hi,I've just touched up my picking code, it was working but not not accurately which I only just noticed. Anyway, what I was doing was treating the position of the camera as the origin of the ray and the direction as (camera - mouse point on the far plane). Changing the origin to the mouse point on the near plane obviously works and using the camera doesn't, my question is why?I would have imagined that as the near plane distance approaches 0 then any points on it approach the camera position, and the only reason we don't set the near plane to be 0 is due to the nonlinearity of the z-buffer.In any case, it works now, im just curious.   
  The camera position is all the same no matter which pixel on the screen your mouse is over, hence a ray constructed from the camera position is all the same no matter which pixel you are clicking on with the mouse. So you really need to use the mouse position because each pixel on the screen has its own ray with its own direction (for perspetive projection at least, for orthogonal the direction would be the same but the position would change as well).The reason is that the camera does not only see a single line but a whole volume that consists of an infinite number of different rays. Of course you are right that the camera position is just a single position but the issue is that you see an image that is projected not onto this singularity of the camera position but onto a projection plane which is an area. Hence the number of rays equals the number of points making up the projection area. In real world this is always an infinite number but in terms of screen resolutions its the pixel width*height of your screen.   ;  Perhaps a fine detail, but is the user selecting or shooting? :)If you're selecting, you usually want to do intersection queries against the line segment mousePosAtNearPlane -> mousePosAtFarPlane (not a ray). If you do the queries against the line segment cameraPos -> mousePosAtFarPlane, you risk picking on objects that are between the camera and the near plane, which are effectively too close to the camera and were culled away in the rendering process. Similarly, you want to avoid querying using a ray, since that extends beyond the far plane and you'd possibly be picking hidden objects as well. Of course if in your setting you know that there won't be any false picks at z distances [0, nearPlaneDist] and [farPlaneDist, +inf] from the camera, then it's not much of a change. Using a finite line segment as opposed to an infinitely long ray is both faster (you won't extend to search through you whole spatial acceleration structure in one direction in the case of a negative result on the query) and more correct (can't pick hidden objects). If you're shooting, it might be more justified to query using a ray that is positioned at cameraPos, and which extends infinitely into the screen. It would also be more accurate to do your queries using the camera position and mouse position from the *previous* frame, which is most probably (modulo triplebuffering) what the user saw when he clicked on the screen. In a standard setting, that's not necessarily much of a difference, but when you have those 13-year old kids that are sending your support people flame e-mails of how they know they really did hit the opponent in the head in your latest FPS.. Sigh..   