title: Apache Spark SQL identifier expected exception
body: <p>My question is quite similar to this one: <a href="https://stackoverflow.com/questions/28410545/apache-spark-sql-issue-java-lang-runtimeexception-1-517-failure-identifier">Apache Spark SQL issue : java.lang.RuntimeException: [1.517] failure: identifier expected</a> But I just can't figure out where my problem lies. I am using SQLite as database backend. Connecting and simple select statements work fine.</p>
  
  <p>The offending line:</p>
  
  <pre><code>val df = tableData.selectExpr(tablesMap(t).toSeq:_*).map(r =&gt; myMapFunc(r))
  </code></pre>
  
  <p>tablesMap contains the table name as key and an array of strings as expressions. Printed, the array looks like this:</p>
  
  <pre><code>WrappedArray([My Col A], [ColB] || [Col C] AS ColB)
  </code></pre>
  
  <p>The table name is also included in square brackets since it contains spaces. The exception I get:</p>
  
  <pre><code>Exception in thread "main" java.lang.RuntimeException: [1.1] failure: identifier expected
  </code></pre>
  
  <p>I already made sure not to use any Spark Sql keywords. In my opinion there are 2 possible reasons why this code fails: 1) I somehow handle spaces in column names wrong. 2) I handle concatenation wrong.</p>
  
  <p>I am using a resource file, CSV-like, which contains the expressions I want to be evaluated on my tables. Apart from this file, I want to allow the user to specify additional tables and their respective column expressions at runtime. The file looks like this:</p>
  
  <pre><code>TableName,`Col A`,`ColB`,CONCAT(`ColB`, ' ', `Col C`)
  </code></pre>
  
  <p>Appartently this does not work. Nevertheless I would like to reuse this file, modified of course. My idea was to map the columns with the expressions from an array of strings, like now, to a sequence of spark columns. (This is the only solution for me I could think of, since I want to avoid pulling in all hive dependecies just for this one feature.) I would introduce a small syntax for my expressions to mark raw column names with a <code>$</code> and some keywords for functions like <code>concat</code> and <code>as</code>. But how could I do this? I tried something like this but it's far far away from even compiling.</p>
  
  <pre><code>def columnsMapFunc( expr: String) : Column = {
      if(expr(0) == '$')
          return expr.drop(1)
      else
          return concat(extractedColumnNames).as(newName)
  }
  </code></pre>