title: Python Manager Dictionary Efficiency
body: <p>I have an object oriented Python program where I am doing certain data operations in each object using multiprocessing. I am storing each object in a common manager dictionary. When I want to update an object, first, I am retrieving the object from the dictionary and after the update I am putting it back. My Class structure is like</p>
  
  <pre><code>from src.data_element import Data_element
  from multiprocessing import freeze_support, Process, Manager
  import pandas as pd
  
  class Data_Obj(Data_element):
  
      def __init__(self, dataset_name,name_wo_fields, fields):
          Data_element.__init__(self, dataset_name, name_wo_fields, fields)
          self.depends=['data_1','data_2'] 
  
      def calc(self,obj_dict_manager):
          data_1=obj_dict_manager['data_1']
          data_2=obj_dict_manager['data_2']
  
          self.df = pd.merge(
                            data_1.df, 
                            data_2.df, 
                               on='week', 
                               suffixes=('', '_y')
                             )[['week','val']]
  
  
  def calculate(obj_dict_manager,data): 
       data_obj=obj_dict_manager[data]    
       data_obj.calc(obj_dict_manager)
       obj_dict_manager[data]=data_obj
  
  
  
  if __name__ == '__main__':
      freeze_support()
      manager=Manager()
      obj_dict_manager=manager.dict() 
      obj_dict_manager=create_empty_objects(obj_dict_manager)
  
      joblist=[]
      for data in obj_dict_manager.keys():
          p=Process(target=calculate, args=(obj_dict_manager,data))
          joblist.append(p)
          p.start()
      for job in joblist:
          job.join() 
  </code></pre>
  
  <p>During these operations, there is a significant time spend on</p>
  
  <pre><code>data_1=obj_dict_manager['data_1']
  data_2=obj_dict_manager['data_2']
  </code></pre>
  
  <p>i.e., the 1 second spend during retrieving the objects from the manager dictionary and the rest of the calculation takes another 1 second.</p>
  
  <p>Is there any way that I can reduce the time spent here? I will be doing thousands of such operations and performance is critical for me.</p>