title: Configure Hive metastore in my local Spark shell
body: <p>I need to configure Hive metastore for use with Spark SQL in spark-shell.</p>
  
  <p>I copied my <code>hive-site.xml</code> to spark/conf folder - it didn't work.</p>
  
  <p>Then tried in spark shell</p>
  
  <pre><code>spark.conf.set("hive.metastore.uris","jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true")
  spark.conf.set("spark.sql.catalogImplementation","hive")
  </code></pre>
  
  <p>But got an error: </p>
  
  <pre><code>scala&gt; spark.conf.set("spark.sql.catalogImplementation","hive")
  org.apache.spark.sql.AnalysisException: Cannot modify the value of a static config: spark.sql.catalogImplementation;
    at org.apache.spark.sql.RuntimeConfig.requireNonStaticConf(RuntimeConfig.scala:155)
    at org.apache.spark.sql.RuntimeConfig.set(RuntimeConfig.scala:41)
    ... 49 elided
  
  </code></pre>
  
  <p>Tried opening spark shell using </p>
  
  <pre><code>spark-shell --conf spark.sql.catalogImplementation=hive hive.metastore.uris=jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true
  </code></pre>
  
  <p>still not able to read </p>
  
  <p>Error:</p>
  
  <pre><code>scala&gt; spark.sql("select * from car_test.car_data_table").show
  org.apache.spark.sql.AnalysisException: Table or view not found: `car_test`.`car_data_table`
  
  </code></pre>
  
  <p>Hive metastore is not getting attached to spark sql.</p>
  
  <p>My hive-site.xml</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8" standalone="no"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;
  
  &lt;property&gt;
    &lt;name&gt;javax.jdo.option.ConnectionURL&lt;/name&gt;
    &lt;value&gt;jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;hive.metastore.warehouse.dir&lt;/name&gt;
    &lt;value&gt;/user/raptor/tmp/hive/warehouse/&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;javax.jdo.option.ConnectionDriverName&lt;/name&gt;
    &lt;value&gt;com.mysql.jdbc.Driver&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;javax.jdo.option.ConnectionUserName&lt;/name&gt;
    &lt;value&gt;HIVE_USER&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;hive.metastore.local&lt;/name&gt;
    &lt;value&gt;true&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
    &lt;name&gt;javax.jdo.option.ConnectionPassword&lt;/name&gt;
    &lt;value&gt;HIVE_PASSWORD&lt;/value&gt;
  &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>spark-env.sh</p>
  
  <pre><code>#!/usr/bin/env bash
  
  export JAVA_HOME=/home/user/Softwares/jdk1.8.0_221/
  export SPARK_LOCAL_IP=127.0.1.1
  export HADOOP_HOME=/home/user/Softwares/hadoop-2.7.3/
  export HIVE_HOME=/home/user/Softwares/apache-hive-2.1.1-bin/
  export SPARK_LOCAL_IP=127.0.1.1
  export SPARK_DIST_CLASSPATH=$HADOOP_HOME/bin/hadoop
  #SPARK_DIST_CLASSPATH=$HADOOP_HOME/share/hadoop/common/lib:$HADOOP_HOME/share/hadoop/common:$HADOOP_HOME/share/hadoop/mapreduce:$HADOOP_HOME/share/hadoop/mapreduce/lib:$HADOOP_HOME/share/hadoop/yarn:$HADOOP_HOME/share/hadoop/yarn/lib:$HADOOP_HOME/share/hadoop/hdfs:$HADOOP_HOME/share/hadoop/hdfs/lib
  export SPARK_HOME=/home/user/Softwares/spark-2.4.4-bin-hadoop-2.7-scala-2.12
  export SPARK_CONF_DIR=${SPARK_HOME}/conf
  export SPARK_LOG_DIR=${SPARK_HOME}/logs
  
  </code></pre>