title: Spark: java.io.IOException: No space left on device
body: <p>Now I am learning how to use spark.I have a piece of code which can invert a matrix and it works when the order of the matrix is small like 100.But when the order of the matrix is big like 2000 I have an exception like this:</p>
  
  <pre><code>15/05/10 20:31:00 ERROR DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file /tmp/spark-local-20150510200122-effa/28/temp_shuffle_6ba230c3-afed-489b-87aa-91c046cadb22
  
  java.io.IOException: No space left on device
  </code></pre>
  
  <p>In my program I have lots of lines like this:</p>
  
  <pre><code>val result1=matrix.map(...).reduce(...)
  val result2=result1.map(...).reduce(...)
  val result3=matrix.map(...)
  </code></pre>
  
  <p>(sorry about that because the code is to many to write there)</p>
  
  <p>So I think when I do this Spark create some new rdds,and in my program Spark creates too many rdds so I have the exception.I am not sure if what I thought is correct.</p>
  
  <p>How can I delete the rdds that I won't use any more?Like result1 and result2?</p>
  
  <p>I have tried rdd.unpersist(), it doesn't work.</p>