title: Unable to filter DataFrame using Window function in Spark
body: <p>I try to use a logical expression based on a window-function to detect duplicate records:</p>
  
  <pre><code>df
  .where(count("*").over(Window.partitionBy($"col1",$"col2"))&gt;lit(1))
  .show
  </code></pre>
  
  <p>this gives in Spark 2.1.1: </p>
  
  <pre><code>java.lang.ClassCastException: org.apache.spark.sql.catalyst.plans.logical.Project cannot be cast to org.apache.spark.sql.catalyst.plans.logical.Aggregate
  </code></pre>
  
  <p>on the other hand, it works if i assign the result of the window-function to a new column and then filter that column:</p>
  
  <pre><code>df
  .withColumn("count", count("*").over(Window.partitionBy($"col1",$"col2"))
  .where($"count"&gt;lit(1)).drop($"count")
  .show
  </code></pre>
  
  <p>I wonder how I can write this without using an temporary column?</p>