title: how to get frames from video in parallel using cv2 & multiprocessing in python
body: <p>I've been working with cv2 &amp; multiprocessing in python, and I finally have a working script that does stuff to the individual frames once they are already in an input queue. However, I wanted to speed up getting the frames into the queue in the first place by using multiple cores, so I tried to use the same multiprocessing approach to read each image into the queue. I can't seem to get this to work though, and I'm not sure why. I thought maybe it was because I was trying to write to one queue, so I split those up, but now I'm wondering if it's because I'm trying to read from the same video file at the same time.</p>
  
  <p>Here is what I am hoping to accomplish in pseudocode:</p>
  
  <pre><code>for process in range(processCount):
      start a process that does this:
          for frame in range(startFrame,endFrame):
              set next frame to startFrame
              read frame
              add frame to queue
  </code></pre>
  
  <p>Here is my current code. I've tried using pool &amp; separate processes, but for now I'm sticking to separate processes because I'm not sure if the problem is with queue management. If I call getFrame manually, I get the right stuff into the queue, so I think that function by itself works okay.</p>
  
  <p>I'm sure I'm doing something really silly (or really odd). Can someone suggest a solution? It would be great to just have one queue as well... I just had two to try to break down the problem.</p>
  
  <p>Thanks in advance.</p>
  
  <pre><code>import numpy as np
  import cv2
  import multiprocessing as mp
  import time
  
  def getFrame(queue, startFrame, endFrame):
      for frame in range(startFrame, endFrame):
          cap.set(1,frame)
          frameNo = int(cap.get(0))
          ret, frame = cap.read()
          queue.put((frameNo,frame))
  
  file = 'video.mov'
  cap = cv2.VideoCapture(file)
  fileLen = int(cap.get(7))
  
  # get cpuCount for processCount
  processCount = mp.cpu_count()/3
  
  inQ1 = mp.JoinableQueue()  # not sure if this is right queue type, but I also tried mp.Queue()
  inQ2 = mp.JoinableQueue()
  qList = [inQ1,inQ2]
  
  # set up bunches
  bunches = []
  for startFrame in range(0,fileLen,fileLen/processCount):
      endFrame = startFrame + fileLen/processCount
      bunches.append((startFrame,endFrame))
  
  getFrames = []
  for i in range(processCount):
      getFrames.append(mp.Process(target=getFrame, args=(qList[i], bunches[i][0],bunches[i][1],)))
  
  for process in getFrames:
      process.start()
  
  results1 = [inQ1.get() for p in range(bunches[0][0],bunches[0][1])]
  results2 = [inQ2.get() for p in range(bunches[1][0],bunches[1][1])]
  
  inQ1.close()
  inQ2.close()
  cap.release()
  
  for process in getFrames:
      process.terminate()
      process.join()
  </code></pre>