title: Adding new Columns based on aggregation on existing column in Spark DataFrame using scala
body: <p>I have a DataFrame like below. I need to create a new column based on existing columns. </p>
  
  <pre><code>col1 col2
  a      1
  a      2
  b      1
  c      1
  d      1
  d      2
  </code></pre>
  
  <p>Output Data Frame look like this </p>
  
  <pre><code>col1  col2 col3 col4
  a      1   1      2
  a      2   1      2
  b      1   0      1
  c      1   0      1
  d      1   1      2
  d      2   1      2
  </code></pre>
  
  <p>The logic I have used to find col3 is <strong>if count of col1 > 1</strong> and <strong>col4 is max value of col2</strong>. </p>
  
  <p>I am familiar with how to do it in sql . But it's hard to find solution with dataframe DSL. Any help would be appreciated. Thanks</p>