Calculate the position and rotation of a quad in 3d space given a 2d projection of that quad from a camera
Hello Djgaven588 and welcome to the forums!
This is not a computer graphics question, but a computer vision one. The answer to it is a little bit involved and I cannot make it justice on a forum post. However, I can point you in the right direction so that you are able to find the answer by yourself.
Here's a helpful link: https://docs.opencv.org/master/d9/db7/tutorial_py_table_of_contents_calib3d.html
The process is somewhat straightforward, but it depends on how much you know about projections and solving systems of equations.
First you need to find key points on a known object in your image through some kind of image-processing algorithm. It seems you already did that. By known object, I mean that you know its exact measurements in the real world with regards to the selected key points.
Secondly, you need to &quot;unproject&quot; your 2D image points onto 3D space, by reversing the projection math from 3D to 2D. This requires building a projection matrix containing intrinsic and extrinsic camera parameters, as well as finding coefficients modelling the camera lens distortion. This process is known as camera calibration.
Thirdly, when using a single camera, you can unproject the points only up to a scale factor, because the depth coordinate has been lost during the original projection. In order to figure out the scale, rotation, and position of the object, you can set up a system of equations relating the image key points to the known key points in the real world at a specific position and rotation of the known object. Solving this system will let you figure out the transform. Note that it is important to have a reasonable amount of key points (even redundant ones) so that the error can be minimised.
OpenCV has methods and functions for doing all of these procedures, and there are many tutorials on the web, like the one I linked above.
I hope you can figure it out! :)