title: Python: Effective storage of data in memory
body: <p>I have the following dictionary structure ( 10,000 keys with values being formed by list of lists )  </p>
  
  <pre><code>my_dic ={0: [[1,.65,3, 0, 5.5], [[4, .55, 3, 0, 5.5] ...(10,000th value)[3,.15, 2, 1,   2.5]], 
  1:[[1,.65,3, 0, 5.5], [[4, .55, 3, 0, 5.5] ...(10,000th value)[3,.15, 2, 1, 2.5]] .....   
  10,000th key:[[1,.65,3, 0, 5.5], [[4, .55, 3, 0, 5.5] ...(10,000th value)[3,.15, 2, 1, 2.5]]}
  </code></pre>
  
  <p>(Note: Data is dummy, so I have just repeated it across the keys)</p>
  
  <p>The logical data types that I want in the smaller elementry list are </p>
  
  <pre><code>inner_list = [int, float, small_int, boolean( 0 or 1), float]
  </code></pre>
  
  <p>A <code>sys.getsizeof(inner_list)</code>, shows it's size to be <code>56</code> bytes. Adding <code>12</code> bytes for the int key makes it <code>68</code> bytes. Now, since I have <code>10^8</code> such lists ( 10000*10000) it's storage in memory is becoming a big problem. I want the data in memory ( no DB as of now ). What should be the most optimized method of storing it ? I am inclined to think that it must have something to do with <code>numpy</code> but not sure on what would be the best method and how to implement it. Any suggestions ?</p>
  
  <p>2) Also, since I am storing these dictionaries in memory, I would like to clear the memory occupied by them as soon as I am done using them. Is there a way of doing this in python? </p>