title: Any tips for making Hive run faster on Hadoop?
body: <p>I'm new to Hive and Hadoop. I have Hadoop configured for pseudo-distributed operation with one data node and one name node, all on localhost.</p>
  
  <p>I have a trivial employee table containing 4 records. I can select the records in a reasonable amount of time, but anything beyond that takes really long. For example:</p>
  
  <pre><code>0: jdbc:hive2://localhost:10000&gt; select * from emp;
  +------------+------------+-------------+-------------+------------+
  | emp.empno  | emp.ename  |   emp.job   | emp.deptno  | emp.etype  |
  +------------+------------+-------------+-------------+------------+
  | 7369       | SMITH      | CLERK       | 10          | PART_TIME  |
  | 7400       | JONES      | ENGINEER    | 10          | FULL_TIME  |
  | 7500       | BROWN      | NIGHTGUARD  | 20          | FULL_TIME  |
  | 7510       | LEE        | ENGINEER    | 20          | FULL_TIME  |
  +------------+------------+-------------+-------------+------------+
  4 rows selected (0.643 seconds)
  0: jdbc:hive2://localhost:10000&gt; select * from emp order by empno;
  WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
  +------------+------------+-------------+-------------+------------+
  | emp.empno  | emp.ename  |   emp.job   | emp.deptno  | emp.etype  |
  +------------+------------+-------------+-------------+------------+
  | 7369       | SMITH      | CLERK       | 10          | PART_TIME  |
  | 7400       | JONES      | ENGINEER    | 10          | FULL_TIME  |
  | 7500       | BROWN      | NIGHTGUARD  | 20          | FULL_TIME  |
  | 7510       | LEE        | ENGINEER    | 20          | FULL_TIME  |
  +------------+------------+-------------+-------------+------------+
  4 rows selected (225.852 seconds)
  </code></pre>
  
  <p>What's it doing that's taking so long? Are there polling periods that I could reduce? I know that Hive isn't optimized for small tasks but this seems ridiculous.</p>
  
  <p>Here's the various files:
  hive-site.xml</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8" standalone="no"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;
    &lt;property&gt;
      &lt;name&gt;hive.mapred.mode&lt;/name&gt;
      &lt;value&gt;nostrict&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;javax.jdo.option.ConnectionURL&lt;/name&gt;
      &lt;value&gt;jdbc:derby:;databaseName=metastore_db;create=true&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;javax.jdo.option.ConnectionDriverName&lt;/name&gt;
      &lt;value&gt;org.apache.derby.jdbc.EmbeddedDriver&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;hive.exec.local.scratchdir&lt;/name&gt;
      &lt;value&gt;/home/hadoop/tmp&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;hive.downloaded.resources.dir&lt;/name&gt;
      &lt;value&gt;/home/hadoop/tmp/${hive.session.id}_resources&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;hive.querylog.location&lt;/name&gt;
      &lt;value&gt;/home/hadoop/tmp&lt;/value&gt;
    &lt;/property&gt;
    &lt;property&gt;
      &lt;name&gt;hive.server2.logging.operation.log.location&lt;/name&gt;
      &lt;value&gt;/home/hadoop/tmp/operation_logs&lt;/value&gt;
    &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>hdfs.xml</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;
     &lt;property&gt;
        &lt;name&gt;dfs.replication&lt;/name&gt;
        &lt;value&gt;1&lt;/value&gt;
     &lt;/property&gt;
     &lt;property&gt;
        &lt;name&gt;dfs.name.dir&lt;/name&gt;
        &lt;value&gt;file:///home/hadoop/hadoopinfra/hdfs/namenode&lt;/value&gt;
     &lt;/property&gt;
     &lt;property&gt;
        &lt;name&gt;dfs.data.dir&lt;/name&gt;
        &lt;value&gt;file:///home/hadoop/hadoopinfra/hdfs/datanode&lt;/value&gt;
     &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>core-site.xml</p>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;
     &lt;property&gt;
        &lt;name&gt;fs.default.name&lt;/name&gt;
        &lt;value&gt;hdfs://localhost:9000&lt;/value&gt;
     &lt;/property&gt;
     &lt;property&gt;
        &lt;name&gt;hadoop.proxyuser.hive.groups&lt;/name&gt;
        &lt;value&gt;*&lt;/value&gt;
     &lt;/property&gt;
     &lt;property&gt;
        &lt;name&gt;hadoop.proxyuser.hive.hosts&lt;/name&gt;
        &lt;value&gt;*&lt;/value&gt;
     &lt;/property&gt;
     &lt;property&gt;
        &lt;name&gt;hadoop.proxyuser.hadoop.hosts&lt;/name&gt;
        &lt;value&gt;*&lt;/value&gt;
     &lt;/property&gt;
     &lt;property&gt;
        &lt;name&gt;hadoop.proxyuser.hadoop.groups&lt;/name&gt;
        &lt;value&gt;*&lt;/value&gt;
     &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>mapred-site.xml</p>
  
  <pre><code>&lt;?xml version="1.0"?&gt;
  &lt;?xml-stylesheet type="text/xsl" href="configuration.xsl"?&gt;
  &lt;configuration&gt;
     &lt;property&gt;
        &lt;name&gt;mapreduce.framework.name&lt;/name&gt;
        &lt;value&gt;yarn&lt;/value&gt;
     &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>
  
  <p>yarn-site.xml</p>
  
  <pre><code>&lt;?xml version="1.0"?&gt;
  &lt;configuration&gt;
     &lt;property&gt;
        &lt;name&gt;yarn.nodemanager.aux-services&lt;/name&gt;
        &lt;value&gt;mapreduce_shuffle&lt;/value&gt;
     &lt;/property&gt;
  &lt;/configuration&gt;
  </code></pre>