title: efficient aggregation (sum) on a single column Data Frame in spark scala
body: <p>I have a Spark Data Frame with a single column and large number of rows (in billions). I am trying to calculate the sum of the values in each row using the code shown below. However, it is very slow. Is there an efficient way to calculate the sum?</p>
  
  <pre><code>val df = sc.parallelize(Array(1,3,5,6,7,10,30)).toDF("colA")
  df.show()
  df.agg(sum("colA")).first().get(0) //very slow 
  </code></pre>
  
  <p>Similar query was posted here: <a href="https://stackoverflow.com/questions/37032025/how-to-sum-the-values-of-one-column-of-a-dataframe-in-spark-scala">How to sum the values of one column of a dataframe in spark/scala</a> 
  The focus of this query is however about efficiency.</p>