title: Cannot change hive.exec.max.dynamic.partitions in Spark
body: <p>I'm trying to insert some data in a table that will have 1500 dynamic partitions and I receive this error:</p>
  
  <pre><code> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
   Number of dynamic partitions created is 1500, which is more than 1000. 
   To solve this try to set hive.exec.max.dynamic.partitions to at least 1500.
  </code></pre>
  
  <p>So, I try to: <code>SET hive.exec.max.dynamic.partitions=2048</code> but I still get the same error. </p>
  
  <p>How can I change this value from Spark? </p>
  
  <p>Code: </p>
  
  <pre><code>this.spark.sql("SET hive.exec.dynamic.partition=true")
  this.spark.sql("set hive.exec.dynamic.partition.mode=nonstrict")
  this.spark.sql("SET hive.exec.max.dynamic.partitions=2048")
  this.spark.sql(
      """
        |INSERT INTO processed_data
        |PARTITION(event, date)
        |SELECT c1,c2,c3,c4,c5,c6,c7,c8,c9,c10,event,date FROM csv_data DISTRIBUTE BY event, date
      """.stripMargin
  ).show()
  </code></pre>
  
  <p>Using Spark 2.0.0 standalone mode. 
  Thank you!</p>