title: OpenCV / Python : multi-threading for live facial recognition
body: <p>I'm using OpenCv and Dlib to execute <strong>facial recognition</strong> w/ landmarks, live from the <strong>webcam stream</strong>. The language is <strong>Python</strong>. It works fine on my macbook laptop, but I need it to run from a desktop computer 24/7. The computer is a PC Intel® Core™2 Quad CPU Q6600 @ 2.40GHz 32bit running Debian Jessie. <strong>The drop in performance is drastic</strong> : there is a 10 seconds delay due to processing !</p>
  
  <p>I therefore looked into <strong>multi-threading</strong> to gain performance :</p>
  
  <ol>
  <li>I first tried the sample code by OpenCv, and the result is great! All four cores hit 100%, and the performance is much better.</li>
  <li>I then replaced the frame processing code with my code, and it doesn't improve performance at all ! Only one core hits the 100%, the other ones stay very low. <strong>I even think it's worse with multi-threading on.</strong></li>
  </ol>
  
  <p>I got the facial landmark code from the dlib sample code. I know it can probably be optimized, but I want to understand <strong>why am I not able to use my (old) computer's full power with multi-threading ?</strong></p>
  
  <p>I'll drop my code below, thanks a lot for reading :) </p>
  
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>from __future__ import print_function
  
  import numpy as np
  import cv2
  import dlib
  
  from multiprocessing.pool import ThreadPool
  from collections import deque
  
  from common import clock, draw_str, StatValue
  import video
  
  class DummyTask:
      def __init__(self, data):
          self.data = data
      def ready(self):
          return True
      def get(self):
          return self.data
  
  if __name__ == '__main__':
      import sys
  
      print(__doc__)
  
      try:
          fn = sys.argv[1]
      except:
          fn = 0
      cap = video.create_capture(fn)
      
      #Face detector
      detector = dlib.get_frontal_face_detector()
  
      #Landmarks shape predictor 
      predictor = dlib.shape_predictor("landmarks/shape_predictor_68_face_landmarks.dat")
  
      # This is where the facial detection takes place
      def process_frame(frame, t0, detector, predictor):
          # some intensive computation...
          gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
          clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
          clahe_image = clahe.apply(gray)
          detections = detector(clahe_image, 1)
          for k,d in enumerate(detections): 
              shape = predictor(clahe_image, d) 
              for i in range(1,68): #There are 68 landmark points on each face
                 cv2.circle(frame, (shape.part(i).x, shape.part(i).y), 1, (0,0,255), thickness=2)
          return frame, t0
  
      threadn = cv2.getNumberOfCPUs()
      pool = ThreadPool(processes = threadn)
      pending = deque()
  
      threaded_mode = True
  
      latency = StatValue()
      frame_interval = StatValue()
      last_frame_time = clock()
      while True:
          while len(pending) &gt; 0 and pending[0].ready():
              res, t0 = pending.popleft().get()
              latency.update(clock() - t0)
              draw_str(res, (20, 20), "threaded      :  " + str(threaded_mode))
              draw_str(res, (20, 40), "latency        :  %.1f ms" % (latency.value*1000))
              draw_str(res, (20, 60), "frame interval :  %.1f ms" % (frame_interval.value*1000))
              cv2.imshow('threaded video', res)
          if len(pending) &lt; threadn:
              ret, frame = cap.read()
              t = clock()
              frame_interval.update(t - last_frame_time)
              last_frame_time = t
              if threaded_mode:
                  task = pool.apply_async(process_frame, (frame.copy(), t, detector, predictor))
              else:
                  task = DummyTask(process_frame(frame, t, detector, predictor))
              pending.append(task)
          ch = cv2.waitKey(1)
          if ch == ord(' '):
              threaded_mode = not threaded_mode
          if ch == 27:
              break
  cv2.destroyAllWindows()</code></pre>
  </div>
  </div>
  </p>