   [solved]cannot setup the pick matrix correctly, please help
  hi guys, i'm working on my simple 3d modeller program, and i'm now implementing the mouse pick function, but i cannot setup the pick matrix correctly, here is current code,void Camera::setCameraForSelection(size_t x1,size_t y1,size_t x2,size_t y2){glViewport((GLint)startX,(GLint)startY,(GLint)width,(GLint)height);GLint viewport[4];glGetIntegerv (GL_VIEWPORT, viewport);glMatrixMode(GL_PROJECTION);glLoadIdentity();gluPickMatrix((GLdouble)x1,(GLdouble)(viewport[3]-y1),(GLdouble)(x2-x1),(GLdouble)( y2-y1),viewport);gluPerspective(angle,width/height,nearPlane,farPlane);glMatrixMode(GL_MODELVIEW);glLoadIdentity();gluLookAt(eye.x,eye.y,eye.z,target.x,target.y,target.z,up.x,up.y,up.z);};the (x1,y1) is the up-left point of the selection area and the (x2,y2) is the low-right point of the area.but when i do selection like this, nothing happened:and when i select in empty area, some faces are selected:i found an example in the opengl redbook, the difference is that the example pick up things when the mouse clicks, so it setups the pick matrix like this:gluPickMatrix((GLdouble) x,(GLdouble) (viewport[3] - y), 5.0, 5.0, viewport);since i need area selection feature, i modified the pick matrix to:gluPickMatrix((GLdouble) x1,(GLdouble) (viewport[3] - y1),(GLdouble)(x2-x1),(GLdouble)( y2-y1), viewport);i do not see any problem, but it just doesn't work correctly.and if i use the example code and change my code to mouse-click pick, it works perfectly. so please help me. thank you very much.[Edited by - billconan on September 7, 2006 2:03:41 PM]
  I'm assuming you know that [0,0] lies at the bottom left corner (the Redbook sample takes that in account, not sure you are aware of that) and you have to flip the cursor Y coordinate.This is a nice trick that I used (from OpenGL FAQ):Quote:20.040 How can I debug my picking code?A good technique for debugging picking or selection code is not to call glRenderMode(GL_SELECT). Simply comment out this function call in your code. The result is instead of performing a selection, your code will render the contents of the pick box to your window. This allows you to see visually what is inside your pick box.Along with this method, it's generally a good idea to enlarge your pick box, so you can see more in your window.Hope that helps.Sweet GUI by the way! :)   ;Quote:Original post by remdulI'm assuming you know that [0,0] lies at the bottom left corner (the Redbook sample takes that in account, not sure you are aware of that) and you have to flip the cursor Y coordinate.This is a nice trick that I used (from OpenGL FAQ):Quote:20.040 How can I debug my picking code?A good technique for debugging picking or selection code is not to call glRenderMode(GL_SELECT). Simply comment out this function call in your code. The result is instead of performing a selection, your code will render the contents of the pick box to your window. This allows you to see visually what is inside your pick box.Along with this method, it's generally a good idea to enlarge your pick box, so you can see more in your window.Hope that helps.Sweet GUI by the way! :)thanks, i thought of render the result on the screen, but the swap buffer function is not in the same class, i will try it anyway. thanks.;  hi there, i've test the selection area by render it out, but i just don't get it. the mouse position is correctly pasd to the pick matrix function, however, the selection area seems to offset up-left. i just don't get it.the blue area is the selection i made and the red one is the pick matrix function generated:this is the scene with the pick matix applied:i still cannot get it. why?   ;  haha, i find out the problem, the first two parameters of the function gluPickMatrix is not the coordinates of the up-left corner of the selection area as i ever assumed. however they are the center point's coordinates of the selection area.so i changed the code togluPickMatrix((GLdouble) (x1+x2)/2,(GLdouble) (viewport[3] - (y1+y2)/2), (GLdouble)(x2-x1),(GLdouble)( y2-y1), viewport);and it works!!!! :)   