title: Spark 2.0-2.3 DataSets groupByKey and mapGroups
body: <p>I see the correct output of the records when I run locally. However, when I run on a cluster the output is different, and seemingly inconsistent. Even some of the mappedGroup output is correct. Is this an issue with spark Closures? Not sure how best to describe what I am seeing.</p>
  
  <p>It is possible I may not understand mapGroups and that not all values for each group are making it to the recordList variable.</p>
  
  <pre><code>case class MyCaseClass (keyValue: int,c2: String,c3: String,c4: Double)
  
  case class NewClass (thing1:String,thing2:String,thing3:String,thing4:String)
  
  case class WorkTodo(myClassRecords: Seq[MyCaseClass]){
      def toNewRecords: Seq[NewClass] = {
      //e.g. work that requires all MyCaseClass.keyValue=1 to be in the list. 
      //This function would create new Java Objects to perform calculations and eventually output a set of NewClass records
  }
  
  val processedRecords = ds.as[MyCaseClass].groupByKey(_.keyValue)
        .mapGroups {
        case (v, iter) =&gt; {
          var recordList = new ListBuffer[MyCaseClass]
          iter.foreach {x=&gt;
            recordList += MyCaseClass(x.keyValue,x.c2,x.c3,x.c4)
          }
  
          WorkToDo(recordList).toNewRecords 
        }
      }
  </code></pre>
  
  <p>P.S. Any other solutions that still use a Dataset are welcome :)</p>