title: How to keep a very large dictionary loaded in memory in python?
body: <p>I have a very large dictionary of size ~ 200 GB which I need to query very often for my algorithm. To get quick results, I want to put it in memory which is possible, because fortunately I have a 500GB RAM. </p>
  
  <p>However, my main issue is that I want to load it only once in memory and then let other processes query the same dictionary, rather than having to load it again everytime I create a new process or iterate over my code.</p>
  
  <p>So, I would like something like this:</p>
  
  <p>Script 1:</p>
  
  <pre><code> # Load dictionary in memory
   def load(data_dir):
       dictionary = load_from_dir(data_dir) ... 
  </code></pre>
  
  <p>Script 2:  </p>
  
  <pre><code> # Connect to loaded dictionary (already put in memory by script 1)
   def use_dictionary(my_query):
       query_loaded_dictionary(my_query) 
  </code></pre>
  
  <p>What's the best way to achieve this ? I have considered a rest API, but I wonder if going over a REST request will erode all the speed I gained by putting the dictionary in memory in the first place.</p>
  
  <p>Any suggestions ?</p>