title: Frequency Distribution of a Column
body: <p>Given a column like so:</p>
  
  <pre><code>+----------+
  | firstName|
  +----------+
  | bob      |
  | jack     |
  | bob      |
  +----------+
  </code></pre>
  
  <p>I want to output the frequency distribution in the following format:
  <strong>bob -> 2</strong>. I'm trying to create a UDF in Scala/ Spark Data frames. The end goal is that given a dataframe, I can output the frequency distribution of each column in the dataframe.  </p>
  
  <pre><code>def freqDist(col: Column): Column = {
   //need help with this part
  }
  
  val freqDist = df.map(ct =&gt; freqDist(col(ct._1))).toList
  //iterate through each column and compute the freqDist like so
  </code></pre>
  
  <p>New Approach:</p>
  
  <pre><code>val freqDist = df.dtypes.map(ct =&gt; {df.select(ct._1).groupBy(col(ct._1)).count()})
  </code></pre>
  
  <p>This gives me the freqDist of each column but the type of freqDist is now an Array of dataframes. How can I aggregate all dataframes into one?</p>