title: DISTRIBUTE BY and CLUSTER BY Not Supported in spark sql 1.6
body: <p>I am using spark 1.6 and and trying to optimize my joins by following these blogs <a href="https://docs.cloud.databricks.com/docs/latest/databricks_guide/04%20SQL,%20DataFrames%20&amp;%20Datasets/09%20Cluster%20By.html" rel="nofollow noreferrer">https://docs.cloud.databricks.com/docs/latest/databricks_guide/04%20SQL,%20DataFrames%20&amp;%20Datasets/09%20Cluster%20By.html</a> and 
  <a href="https://blog.deepsense.ai/optimize-spark-with-distribute-by-and-cluster-by/" rel="nofollow noreferrer">https://blog.deepsense.ai/optimize-spark-with-distribute-by-and-cluster-by/</a>  using DISTRIBUTE BY and CLUSTER BY , but unfortunately they are not supported. </p>
  
  <p>My spark sql query is </p>
  
  <pre><code>sqlContext.sql(
        """select b.*, count(*) AS CNT  from tableb b
           GROUP BY b.Key,b.KeyVal
           CLUSTER BY b.Key,b.KeyVal
        """)
  </code></pre>
  
  <p>Error is </p>
  
  <pre><code>Exception in thread "main" java.lang.RuntimeException: [5.7] failure: ``union'' expected but identifier CLUSTER found
  
        CLUSTER BY b.Key
  </code></pre>