title: Repairing partition data in hdfs
body: <p>Every hour data is added to the hdfs partition - (date, hour) in the form of tsv files. An external table 'logs' has that hdfs location - 'hive/log_data'.
  Divided in partitions, for instance - 'hive/log_data/utc_date=2016-08-31/utc_hour=15'</p>
  
  <p>After which a command is run </p>
  
  <pre><code>hive -e "ALTER TABLE logs ADD IF NOT EXISTS PARTITION (utc_date='${DATE}', utc_hour='${HOUR}')"
  </code></pre>
  
  <p>For that particular partition so that partitions created in the hive metastore - just like msck repair table logs. 
  Then a script is run to compress the file so as to not have too many small files. </p>
  
  <pre><code>SET hive.exec.compress.output=true;
  insert overwrite into table PARTITION( utc_date = '${hiveconf:DATE}',
  utc_hour = '${hiveconf:HOUR}' 
  select * from logs
  where utc_date = '${hiveconf:DATE}'
  and utc_hour = '${hiveconf:HOUR}'
  </code></pre>
  
  <p>For one case, ALTER TABLE add partition command was missed for all hours of the day. The problem happened here when this compress script ran on partitions, information of which wasn't present in the hive metastore.</p>
  
  <p>Now, for the logs table any query that queries that table for that date it is really slow and creates 1000 times more mappers compared to other dates. </p>
  
  <p>What can be done to repair the partition. Running ALTER or msck command now doesn't help even though it completes succesfully. </p>