title: Python multiprocessing progress approach
body: <p>I've been busy writing my first multiprocessing code and it works, yay.
  However, now I would like some feedback of the progress and I'm not sure what the best approach would be.</p>
  <p>What my code (see below) does in short:</p>
  <ul>
  <li>A target directory is scanned for mp4 files</li>
  <li>Each file is analysed by a separate process, the process saves a result (an image)</li>
  </ul>
  <p>What I'm looking for could be:</p>
  <ol>
  <li>Simple</li>
  </ol>
  <ul>
  <li>Each time a process finishes a file it sends a 'finished' message</li>
  <li>The main code keeps count of how many files have finished</li>
  </ul>
  <ol start="2">
  <li>Fancy</li>
  </ol>
  <pre><code>Core 0  processing file 20 of 317 ||||||____ 60% completed
  Core 1  processing file 21 of 317 |||||||||_ 90% completed
  ...
  Core 7  processing file 18 of 317 ||________ 20% completed
  </code></pre>
  <p>I read all kinds of info about queues, pools, tqdm and I'm not sure which way to go. Could anyone point to an approach that would work in this case?</p>
  <p>Thanks in advance!</p>
  <p>EDIT: Changed my code that starts the processes as suggested by gsb22</p>
  <p>My code:</p>
  <pre><code># file operations
  import os
  import glob
  # Multiprocessing
  from multiprocessing import Process
  # Motion detection
  import cv2
  
  
  # &gt;&gt;&gt; Enter directory to scan as target directory
  targetDirectory = &quot;E:\Projects\Programming\Python\OpenCV\\videofiles&quot;
  
  def get_videofiles(target_directory):
  
      # Find all video files in directory and subdirectories and put them in a list
      videofiles = glob.glob(target_directory + '/**/*.mp4', recursive=True)
      # Return the list
      return videofiles
  
  
  def process_file(videofile):
  
      '''
      What happens inside this function:
      - The video is processed and analysed using openCV
      - The result (an image) is saved to the results folder
      - Once this function receives the videofile it completes
        without the need to return anything to the main program
      '''
  
      # The processing code is more complex than this code below, this is just a test
      cap = cv2.VideoCapture(videofile)
  
      for i in range(10):
          succes, frame = cap.read()
  
          # cv2.imwrite('{}/_Results/{}_result{}.jpg'.format(targetDirectory, os.path.basename(videofile), i), frame)
  
          if succes:
              try:
                  cv2.imwrite('{}/_Results/{}_result_{}.jpg'.format(targetDirectory, os.path.basename(videofile), i), frame)
              except:
                  print('something went wrong')
  
  
  if __name__ == &quot;__main__&quot;:
  
      # Create directory to save results if it doesn't exist
      if not os.path.exists(targetDirectory + '/_Results'):
          os.makedirs(targetDirectory + '/_Results')
  
      # Get a list of all video files in the target directory
      all_files = get_videofiles(targetDirectory)
  
      print(f'{len(all_files)} video files found')
  
      # Create list of jobs (processes)
      jobs = []
  
      # Create and start processes
      for file in all_files:
          proc = Process(target=process_file, args=(file,))
          jobs.append(proc)
  
      for job in jobs:
          job.start()
  
      for job in jobs:
          job.join()
  
      # TODO: Print some form of progress feedback
  
      print('Finished :)')
  </code></pre>