title: Spark subtractByKey increases RDD cached memory size
body: <p>I've found a very strange behavior for Apache Spark RDD's (spark 1.6.0 with scala 2.11):</p>
  
  <p>When i use subtractByKey on an RDD the resulting RDD should be of equal or smaller size. What i get is an RDD that takes even more space in memory:</p>
  
  <pre><code>//Initialize first RDD
  val rdd1 = sc.parallelize(Array((1,1),(2,2),(3,3))).cache()
  
  //dummy action to cache it =&gt; size according to webgui: 184 Bytes
  rdd1.first
  
  //Initialize RDD to subtract (empty RDD should result in no change for rdd1)
  val rdd2 = sc.parallelize(Array[(Int,Int)]())
  
  //perform subtraction
  val rdd3 = rdd1.subtractByKey(rdd2).cache()
  
  //dummy action to cache rdd3 =&gt; size according to webgui: 208 Bytes
  rdd3.first
  </code></pre>
  
  <p>I frist realized this strange behaviour for an RDD of ~200k rows and size 1.3 GB that scaled up to more than 2 GB after subtraction</p>
  
  <p>Edit: Tried the example above with more values(10k) => same behaviour. The size increases by ~1.6 times. Also reduceByKey seems to have a similar effect. </p>
  
  <p>When i create an RDD by</p>
  
  <pre><code>sc.paralellize(rdd3.collect())
  </code></pre>
  
  <p>the size is the same as for <em>rdd3</em>, so the increased size carries over even if it's extracted from RDD. </p>