title: How to integrate tqdm with python multiprocessing
body: <p>I am creating a new python class where I am trying to integrate multiprocessing as well as tqdm to illustrate progress.  I am going down this path because I am opening very large (>1GB) time series data files, loading into pandas, doing a groupby and then saving them in parquet format. Each datafile can take minutes to process and save.  Multiprocessing speeds up the process immensely. However, I have no visibility currently on the process and I am trying to integrate tqdm.  </p>
  
  <p>The code below illustrates a simple example.  In this code tqdm just shows how long it takes the processes to be allocated to a pool, but does not update per the actual process.</p>
  
  <p>'''python</p>
  
  <pre><code>import time
  import multiprocessing
  from tqdm import tqdm
  
  
  class test_multiprocessing(object):
  
      def __init__(self, *args, **kwargs):
          self.list_of_results=[]
          self.items = [0,1,2,3,4,5,6,7,8,9,10]
  
  
      def run_test(self):
          print(f'Startng test')
  
          for i in range(1,5,1):
              print(f'working on var1: {i}')
  
              p = multiprocessing.Pool()
  
              for j in tqdm(self.items, desc='Items', unit='items', disable=False):
                  variable3=3.14159
                  p.apply_async(self.worker, [i, j,variable3], callback=self.update)
  
              p.close()
              p.join()
              print(f'completed i = {i}')
              print(f'')
  
      def worker(self, var1, var2, var3):
          result=var1*var2*var3
          time.sleep(2)
          return result
  
      def update(self, result_to_save):
          self.list_of_results.append(result_to_save)
  
  if __name__ == '__main__':
      test1=test_multiprocessing()
      test1.run_test()
  </code></pre>
  
  <p>'''</p>
  
  <p>In this example the progress bar will show the work is complete almost immediately, but in reality it takes seconds</p>