title: Spark SQL: get the value of a column when another column is max value inside a groupBy().agg()
body: <p>I have a dataframe that looks like this:</p>
  
  <pre><code>root
   |-- value: int (nullable = true)
   |-- date: date (nullable = true)
  </code></pre>
  
  <p>I'd like to return value where value is the latest date in the dataframe.
  Does this problem change if I need to make a groupBy and agg?
  My actual problem looks like this:</p>
  
  <pre><code>val result = df
  .filter(df("date")&gt;= somedate &amp;&amp; df("date")&lt;= some other date)
  .groupBy(valueFromColumn1)
  .agg(
      max(date),
      min(valueFromColumn2),
      Here I want to put valueFromColumn4 where date is max after the filter
   )
  </code></pre>
  
  <p>I know I can get these values by creating a second dataframe and then making a join. But I'd like to avoid the join operation if possible.</p>
  
  <p>Input sample:</p>
  
  <pre><code>Column 1 | Column 2 | Date | Column 4
      A         1       2006      5
      A         5       2018      2
      A         3       2000      3
      B         13      2007      4
  </code></pre>
  
  <p>Output sameple (filter is date >= 2006, date &lt;= 2018):</p>
  
  <pre><code>Column 1 | Column 2 | Date | Column 4
      A         1       2018      2  &lt;- I got 2 from the first row which has the highest date
      B         13      2007      4
  </code></pre>