title: hive performance union all
body: <p>I have multiple files, combining them through single file using Hive <code>union all</code>, now the <code>union all</code> query joins more than 10 Hive tables, and is very slow. Each individual <code>union all</code> query has one or more join condition as well. </p>
  
  <p>My expected end result is also required to be grouped by primary key, e.g: </p>
  
  <pre><code>table 1 
  
  key1|val1|val2|..
  
  table 2
  
  key1|val10|val11|..
  
  insert overwrite &lt;temptable&gt;
  select key, output_string from
  (select key, concat (col1,col2,..) from table 1 where &lt;join conditions&gt;
  union all
  select key, concat(col10,col11,..) from table 2 where &lt;join conditions&gt;
  ..
  )
  cluster by key;
  </code></pre>
  
  <p><code>union all</code> will combine above two tables, i'm using cluster by key to produce desired result. The performance of hive is extremely slow, what are the other alternatives?</p>