title: Shuffled vs non-shuffled coalesce in Apache Spark
body: <p>What is the difference between the following transformations when they are executed right before writing RDD to a file?</p>
  
  <ol>
  <li>coalesce(1, shuffle = true)</li>
  <li>coalesce(1, shuffle = false)</li>
  </ol>
  
  <p>Code example:</p>
  
  <pre><code>val input = sc.textFile(inputFile)
  val filtered = input.filter(doSomeFiltering)
  val mapped = filtered.map(doSomeMapping)
  
  mapped.coalesce(1, shuffle = true).saveAsTextFile(outputFile)
  vs
  mapped.coalesce(1, shuffle = false).saveAsTextFile(outputFile)
  </code></pre>
  
  <p>And how does it compare with collect()? I'm fully aware that Spark save methods will store it with HDFS-style structure, however I'm more interested in data partitioning aspects of collect() and shuffled/non-shuffled coalesce().</p>