title: AWS Athena insert into with named columns not working in pyspark
body: <p>I've created a little test table using pyspark</p>
  
  <pre><code>query="""
  CREATE EXTERNAL TABLE IF NOT EXISTS test1
  (
  c1 INT,
  c2 INT,
  c3 INT
  )
  ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
  LOCATION 's3://mybucket/myfolder/'
  """
  spark.sql(query)
  </code></pre>
  
  <p>And this works fine , producing the following output</p>
  
  <pre><code>spark.sql("select * from test1").show()
  
  +---+---+---+
  | c1| c2| c3|
  +---+---+---+
  |  1|  2|  3|
  |  4|  5|  6|
  +---+---+---+
  </code></pre>
  
  <p>My problem is trying to do an insert now. According to my reading of the Athena documentation I should be able to do the following but I'm getting an error message</p>
  
  <pre><code>query="""
  insert into test1(c1,c2,c3) select c1,c2,c3 from test1
  """
  spark.sql(query)
  
  
  "\nmismatched input 'c1' expecting {'(', 'SELECT', 'FROM', 'VALUES', 'TABLE', 'INSERT', 'MAP', 'REDUCE'}(line 2, pos 21)\n\n== SQL ==\n\ninsert into test1(c1,c2,c3) select c1,c2,c3 from test1\n---------------------^^^\n"
  Traceback (most recent call last):
    File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/session.py", line 767, in sql
      return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped)
    File "/usr/lib/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__
      answer, self.gateway_client, self.target_id, self.name)
    File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 73, in deco
      raise ParseException(s.split(': ', 1)[1], stackTrace)
  pyspark.sql.utils.ParseException: "\nmismatched input 'c1' expecting {'(', 'SELECT', 'FROM', 'VALUES', 'TABLE', 'INSERT', 'MAP', 'REDUCE'}(line 2, pos 21)\n\n== SQL ==\n\ninsert into test1(c1,c2,c3) select c1,c2,c3 from test1\n---------------------^^^\n"
  </code></pre>
  
  <p>However the following INSERT works as expected</p>
  
  <pre><code>query="""
  insert into test1 select c1,c2,c3 from test1
  """
  
  spark.sql(query)
  </code></pre>
  
  <p>If anyone can see what I'm doing wrong it would be appreciated</p>