title: Spark on embedded mode - user/hive/warehouse not found
body: <p>I'm using Apache Spark in embedded local mode. I have all the dependencies included in my <em>pom.xml</em> and in the same version (spark-core_2.10, spark-sql_2.10, and spark-hive_2.10).</p>
  
  <p>I just want to run a HiveQL query to create a table (stored as Parquet).</p>
  
  <p>Running the following (rather simple) code:</p>
  
  <pre><code>public class App {
      public static void main(String[] args) throws IOException, ClassNotFoundException {
  
          SparkConf sparkConf = new SparkConf().setAppName("JavaSparkSQL").setMaster("local[2]").set("spark.executor.memory", "1g");
          JavaSparkContext ctx = new JavaSparkContext(sparkConf);
          HiveContext sqlContext = new org.apache.spark.sql.hive.HiveContext(ctx.sc());
  
          String createQuery = "CREATE TABLE IF NOT EXISTS Test (id int, name string) STORED AS PARQUET";
          sqlContext.sql(createQuery);
      }
  }
  </code></pre>
  
  <p>...is returning the following exception:</p>
  
  <pre><code>FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:file:/user/hive/warehouse/test is not a directory or unable to create one)
  </code></pre>
  
  <p>I can see the <code>metastore_db</code> folder created in the root of the project.</p>
  
  <p>I searched around and the solutions found didn't help --most of them were not for the embedded mode. </p>
  
  <ul>
  <li>One solution was to check the permissions, I'm using the same user for everything.</li>
  <li>Another solution was to create the folder manually in HDFS, I did and I can navigate to /user/hive/warehouse/test.</li>
  <li>One solution was to set manually the metastore by adding:     <code>sqlContext.sql("SET hive.metastore.warehouse.dir=hdfs://localhost:9000/user/hive/warehouse");</code>.</li>
  </ul>
  
  <p>I'm running out of ideas right now, can someone provide any other suggestions?</p>