title: How to append new data to pickle file using python
body: <p>I am extracting face embedding of an image and appending it in a existing pickle file. But looks like its not working as when I unpickle the file, it do not contains the new data added. Below is code:</p>
  
  <pre><code>file = client_dir + '\embeddings.pickle'
  data = {"embeddings": known_embeddings, "names": known_names}
  with open(file, 'ab+') as fp:
      pickle.dump(data, fp)
      fp.close()
  log("[INFO] Data appended to embeddings.pickle ")
  </code></pre>
  
  <p>Current pickle file contains below data:</p>
  
  <pre><code>{'embeddings': [array([-0.03656099,  0.11354745, -0.00438912,  0.0367547 ,  0.06391761,
          0.18440282,  0.06150107, -0.17380905,  0.03094344, -0.00182147,
          0.00969766,  0.06890091,  0.04974053, -0.0502388 , -0.03414046,
         -0.13550822, -0.02251128,  0.14556041, -0.04045469,  0.06500552,
          0.0726142 , -0.04139924, -0.04662199,  0.08869533, -0.00061307,
         -0.11912274,  0.13141112, -0.00648551,  0.00296356,  0.03682912,
         -0.15076959,  0.03989822,  0.02799555,  0.03429572,  0.09865954,
          0.14113557, -0.08355764,  0.09193961, -0.00819231, -0.01184336,
         -0.12519744,  0.00668721,  0.0816237 ,  0.00464355, -0.00339399,
          0.07501812,  0.11679655, -0.09211859,  0.06211261, -0.00543289,
          0.10347278,  0.06651585, -0.01512023,  0.09477805,  0.09886038,
         -0.03837246,  0.02265131, -0.14867221,  0.00781244,  0.04845129,
         -0.0363168 , -0.00186919, -0.16163988,  0.09539618,  0.14983718,
          0.09159472, -0.05315595, -0.05073383,  0.01501674, -0.03789762,
          0.07116041,  0.07650694, -0.02975985], dtype=float32)], 'names': ['rock']} 
  </code></pre>
  
  <p>New data which I am trying to append is below:</p>
  
  <pre><code>{'embeddings': [array([-0.03656099,  0.11354745, -0.00438912,  0.0367547 ,  0.06391761,
          0.18440282,  0.06150107, -0.17380905,  0.03094344, -0.00182147,
          0.00969766,  0.06890091,  0.04974053, -0.0502388 , -0.03414046,
          0.07501812,  0.11679655, -0.09211859,  0.06211261, -0.00543289,
          -0.13550822, -0.02251128,  0.14556041, -0.04045469,  0.06500552,
          0.0726142 , -0.04139924, -0.04662199,  0.08869533, -0.00061307,
         -0.11912274,  0.13141112, -0.00648551,  0.00296356,  0.03682912,
         -0.15076959,  0.03989822,  0.02799555,  0.03429572,  0.09865954,
          0.14113557, -0.08355764,  0.09193961, -0.00819231, -0.01184336,
         -0.12519744,  0.00668721,  0.0816237 ,  0.00464355, -0.00339399,
          0.10347278,  0.06651585, -0.01512023,  0.09477805,  0.09886038,
         -0.03837246,  0.02265131, -0.14867221,  0.00781244,  0.04845129,
         -0.0363168 , -0.00186919, -0.16163988,  0.09539618,  0.14983718,
          0.09159472, -0.05315595, -0.05073383,  0.01501674, -0.03789762,
          0.07116041,  0.07650694, -0.02975985], dtype=float32)], 'names': ['john']}
  </code></pre>
  
  <p>But when I unpickle the file it only has the the data for <code>rock</code> and not the <code>john</code>. Can anyone please help me what I am doing wrong. Below is the code I am using to unpickle and watch what data is added. May be the way I am unpickling the file is wrong, because when I am appending the data I can see the file size increasing.</p>
  
  <pre><code>import pickle
  
  file = open('G:\\output\\embeddings.pickle', 'rb')
  
  data = pickle.load(file)
  
  file.close()
  
  print(data)
  </code></pre>
  
  <p>Please help. Thanks</p>
  
  <p>Updated code:</p>
  
  <pre><code>file_path = client_dir + '\embeddings.pickle'
  file = open(file_path, 'rb')
  old_data = pickle.load(file)
  new_embeddings = old_data['embeddings']
  new_names = old_data['names']
  new_embeddings.append(known_embeddings[0])
  new_names.append(known_names[0])
  data1 = {"embeddings": new_embeddings, "names": new_names}
  with open(file_path, 'ab+') as fp:
      pickle.dump(data1, fp)
      fp.close()
  log.error("[INFO] Data appended to embeddings.pickle ")
  </code></pre>
  
  <p>In the above code, I am first loading the data from the pickle file into list and then appending the new data into the list and then adding all the data (old + new) into the pickle file. Can anyone please tell me if this is the correct way of doing it. </p>
  
  <p>After this as well, when I unpickle the file, I am not getting all the data. Thanks</p>