title: Scala - how do I convert from DataFrame to Array[caseclass]
body: <p>I am using DataBricks to process data.</p>
  <p>One of the columns in my DataFrame is an Array. I want to extract it so I can process it, but am having problems with the syntax. I seem to be extracting it still as a DataFrame and not converting to an Array</p>
  <p>Here is an example. My data is a set of animals. Each animal has a name and a set of movements.</p>
  <p>I define two Case Classes to hold the data</p>
  <pre><code>case class Movement ( 
    location: String,
    direction: String
  )
  
  case class Animal(
    var name: String,
    var movements: Array[Movement]
  )
  </code></pre>
  <p>I define data for 2 animals, Gordon and David</p>
  <pre><code>val m1 = Movement( &quot;farm1&quot;, &quot;arrive&quot;);
  val m2 = Movement( &quot;farm1&quot;, &quot;leave&quot;);
  val m3 = Movement( &quot;farm2&quot;, &quot;arrive&quot;);
  val m4 = Movement( &quot;farm3&quot;, &quot;arrive&quot;);
  
  val am = Array( m1, m2, m3);
  val am2 = Array( m1, m2, m4);
  val df : Animal = Animal(&quot;Gordon&quot;, am )
  val df2 : Animal = Animal( &quot;David&quot;, am2 )
  val df3 = Seq( df, df2 ).toDF;
  </code></pre>
  <p>I have a routine that processes movements. To simplify things, for this example it just displays them</p>
  <pre><code>def showMoves( amIn: Array[Movement]) {
    for( mv &lt;- amIn ) (
      println( mv.location + &quot; &quot; + mv.direction )
    )
  }
  </code></pre>
  <p>This works fine for Gordon and David's movements</p>
  <pre><code>showMoves( am )
  showMoves( am2 )
  </code></pre>
  <p>And the results are:</p>
  <pre><code>farm1 arrive
  farm1 leave
  farm2 arrive
  
  farm1 arrive
  farm1 leave
  farm3 arrive
  </code></pre>
  <p>This is the data all set up.</p>
  <p>Now I decide I am only interested in Gordon, and want to extract his movements</p>
  <pre><code>val df4 = df3.filter( &quot;name == 'Gordon'&quot;)
  var am3 = df4.select(&quot;movements&quot;).as[Array[Movement]]
  </code></pre>
  <p>Scala tells me:</p>
  <pre><code>am3:org.apache.spark.sql.Dataset[Array[Movement]] = [movements: array]
  </code></pre>
  <p>I now want to process his movements, and here is where the problem is.</p>
  <pre><code>showMoves( am3 )
  </code></pre>
  <p>Scala tells me this</p>
  <pre><code>command-721439694904705:44: error: type mismatch;
   found   : org.apache.spark.sql.Dataset[Array[Movement]]
   required: Array[Movement]
  showMoves( am3 )
  </code></pre>
  <p>So I understand from this I have created am3 as a Dataset[Array[Movement]] instead of Array[Movement], but what I can't work out is how to create am3 in the correct type.</p>