title: What is the overview of serialization in Spark
body: <p>Perhaps someone who has more experience with Spark can help with the following serialization related questions: :)</p>
  
  <p>Would the following be an accurate description of one of the serialization paths that happen in Spark:</p>
  
  <ul>
  <li><code>RDD --&gt; Bytestream (Java/Kryo)</code></li>
  <li><code>Dataset --&gt; Bytestream (Encoders)</code></li>
  </ul>
  
  <p>In this <a href="https://youtu.be/Ofk7G3GD9jk?t=1275" rel="nofollow noreferrer">talk</a> about <code>RDD</code>, <code>Dataframe</code> and <code>Dataset</code> in spark, it was stated that <code>RDD</code> is still the fundamental data structure at play in Spark, and hence even <code>Dataset</code>s builds on <code>RDD</code></p>
  
  <p>How should that statement be interpreted? Does it mean that the serialization path for a <code>Dataset</code> is:</p>
  
  <p><code>Dataset --&gt;(encoders) RDD --&gt; Bytestream(Java/kryo)</code></p>
  
  <p>Or does encoders also work in translating RDD's into Bytestream?</p>