Image distortion and alignment on 3D mesh / pointcloud
If you're only left with matching 3D points to their corresponding 2D parts, this is a fairly task and called PnP pose estimation. OpenCV has a lot of options to solve this problem. Here is one tutorial, which you could maybe benefit.