title: How to define a custom aggregation function to sum a column of Vectors?
body: <p>I have a DataFrame of two columns, <code>ID</code> of type <code>Int</code> and <code>Vec</code> of type <code>Vector</code> (<code>org.apache.spark.mllib.linalg.Vector</code>).</p>
  
  <p>The DataFrame looks like follow:</p>
  
  <pre><code>ID,Vec
  1,[0,0,5]
  1,[4,0,1]
  1,[1,2,1]
  2,[7,5,0]
  2,[3,3,4]
  3,[0,8,1]
  3,[0,0,1]
  3,[7,7,7]
  ....
  </code></pre>
  
  <p>I would like to do a <code>groupBy($"ID")</code> then apply an aggregation on the rows inside each group by summing the vectors.</p>
  
  <p>The desired output of the above example would be:</p>
  
  <pre><code>ID,SumOfVectors
  1,[5,2,7]
  2,[10,8,4]
  3,[7,15,9]
  ...
  </code></pre>
  
  <p>The available aggregation functions will not work, e.g. <code>df.groupBy($"ID").agg(sum($"Vec")</code> will lead to an ClassCastException.</p>
  
  <p>How to implement a custom aggregation function that allows me to do the sum of vectors or arrays or any other custom operation?</p>