title: How do I align a face for preprocessing?
body: <p>I am trying to write a face aligner for as a preprocessing step before feeding my face into a neural network. I have found (using Python) the landmarks of a face via <code>dlib</code>'s implementation of <a href="http://www.csc.kth.se/~vahidk/papers/KazemiCVPR14.pdf" rel="nofollow noreferrer">Vahid Kazemi and Josephine Sullivan's</a> ensemble of regression trees to predict face landmark estimation. I have a tried a number of different kinds of transforms to no avail. This is my basic pipeline:</p>
  
  <ol>
  <li><p>Find faces in image and crop image to only having the face :</p>
  
  <pre><code>image = &lt;some_image_with_one_face&gt;
  face_detector = dlib.get_frontal_face_detector()
  detected_face = face_detector(image)
  w,h = 160 # so that facenet can use it 
  for face_rect in detected_face:
  
      # First crop the face
      left = face_rect.left()
      top = face_rect.top()
      right = face_rect.right()
      bottom = face_rect.bottom()
  
      new_face_rect = dlib.rectangle(0, 0, right-left, bottom-top)
      cropped_image = image[top:bottom, left:right, :].copy()
  
      # Get the the face's pose
      pose_landmarks = face_pose_predictor(cropped_image, new_face_rect)`
  </code></pre></li>
  <li><p>Find the landmarks (<code>get_landmark_points</code> returns a <code>68x2</code> array</p>
  
  <pre><code>landmarks = get_landmark_points(pose_landmarks, N_LANDMARKS, dlib_point=False)
  </code></pre></li>
  <li><p>Define some <code>src</code> and <code>dst</code> points:</p>
  
  <pre><code>top_of_nose = landmarks[27]
  left_eye = landmarks[36]
  right_eye = landmarks[45]
  bottom_lip = landmarks[57]
  
  src = [top_of_nose, left_eye, right_eye, bottom_lip]
  dst = [[np.int(0.5 * w), np.int(h/3)],\
         [np.int(0.3 * w), np.int(h / 3)],\
         [np.int(0.7 * w), np.int(h / 3)],\
         [np.int(0.5 * w), np.int(h * (2.0/3))]]
  </code></pre></li>
  <li><p>Estimate the transform and also transform the rest of the landmarks</p>
  
  <pre><code>transformed_crop = cv2.warpAffine(cropped_image, transform, (w, h))
  
  # Get the transformed landmarks
  transformed_landmarks = np.reshape(cv2.transform(np.expand_dims(landmarks, 1),\
                          transform), (N_LANDMARKS, 2)).astype(np.float32)
  
  # Add the boundary points
  transformed_landmarks = np.append(transformed_landmarks, boundary_points, axis = 0)
  </code></pre></li>
  </ol>
  
  <p>Here is input and outputs of 2 images:
  <a href="https://i.stack.imgur.com/anXoP.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/anXoP.jpg" alt="enter image description here"></a></p>
  
  <p>Although it kind of works, it isn't perfect as although the eyes and lip are of the same height in both images, their <code>x</code> position is different. I was wondering if there is some way to improve it, or am I using the wrong technique? It would be helpful if someone pointed me in the right direction, thanks!</p>