title: How to detect the direction of a finger pointing
body: <p>I am using python with OpenCV and pyautogui to play the game tetris (via a website) with your hands. In order to play the game there are 4 directions (up/down/left/right) that I want to detect to match with inputs for the pyautogui. I have tried so many ways to detect the direction of the finger that it pointing but nothing works consistently,any help on the current solution I am working on, or another idea would be great.</p>
  <p>I am currently trying to use findContours with convexHull to detect the perimeter of the hand and attempt to find the sides with the smallest angle which in turn should tell me the side the finger is pointing. But I don't know how to extract that information.</p>
  <p>I have tried using a BoundingRect to find which side is longer (w or h) then splitting the rectangle in half and seeing which side has more pixels in it (from the binary of the contour), but it wasn't consistent enough.</p>
  <p>I have also tried using HoughLines and HoughLinesP to find a vanishing point, but I couldn't figure out how to get that to work.</p>
  <p>Basically I am at a loss.</p>
  <p>Edit: I'm trying to avoid MediaPipe so I can show I understand how to use the elements of openCV</p>