title: Issues with Spark Serialization?
body: <p>Let be a job that contains two phases that (for convenience) cannot be merged. Let's call <code>A</code> the first step and <code>B</code> the second step. Hence, we always need to do <code>A</code> then <code>B</code>.</p>
  
  <p><strong>Workflow</strong></p>
  
  <ul>
  <li>start new cluster with a job <code>A</code></li>
  <li>build <code>MyOutput</code></li>
  <li><code>count(MyOutput)</code> = 2000 (*)</li>
  <li><code>write(MyOutput)</code></li>
  <li>start new cluster with a job <code>B</code></li>
  <li><code>read(MyOutput)</code></li>
  <li><code>count(MyOutput)</code> = 1788 (**).</li>
  </ul>
  
  <p><strong>Precisions</strong></p>
  
  <ul>
  <li><p><code>A</code> provides an output which is an <code>RDD[MyObject]</code>, namely <code>MyOutput</code>. To write <code>MyOutput</code>, here is what I do: <code>MyOutput.saveAsObjectFile("...")</code></p></li>
  <li><p>Then <code>B</code> uses <code>MyOutput</code> as an input, reading the previously written file. Here is what I do: <code>val MyOutput: RDD[MyObject] = sc.objectFile("...")</code></p></li>
  <li><p><code>A</code> and <code>B</code> happen on two separate clusters.</p></li>
  </ul>
  
  <p><strong>Problem</strong></p>
  
  <p>The main point is that the problem does not ALWAYS happen but when this problem appears, <code>(*) &lt; (**)</code> -- it seems we lost some data whereas we should not have any differences between these data. Obvisously something is wrong.</p>
  
  <p>Do you know what happens here? How to solve that?</p>