title: How do you add Data to an Existing Hive Metastore?
body: <p>I have multiple subdirectories in S3 that contain .orc files.  I'm trying to create a hive metastore so I can query the data with Presto / Hive, etc.  The data is poorlly structured (no consistent delimiter, ugly characters, etc).  Here's a scrubbed sample:</p>
  
  <pre><code>1488736466 199.199.199.199 0_b.www.sphericalcow.com.f9b1.qk-g6m6z24tdr.v4.url.name.com TXT IN: NXDOMAIN/0/143
  1488736466 6.6.5.4 0.3399.186472.4306.6668.638.cb5a.names-things.update.url.name.com TXT IN: NOERROR/3/306 0\009253\009http://az.blargi.ng/%D3%AB%EF%BF%BD%EF%BF%BD/\009 0\009253\009http://casinoroyal.online/\009 0\009253\009http://d2njbfxlilvpsq.cloudfront.net/b_zq_ym_bangvideo/bangvideo0826.apk\009 
  </code></pre>
  
  <p>I was able to create a table pointing to one of the subdirectories using a serde regex and the fields are parsing properly, but as far as I can tell I can only load one subfolder at a time.</p>
  
  <p>How does one add more data to an existing hive metastore?</p>
  
  <p>Here's an example of my hive metastore create statement with the regex serde bit:</p>
  
  <pre><code>DROP TABLE IF EXISTS test;
  
  CREATE EXTERNAL TABLE test (field1 string, field2 string, field3 string, field4 string)
  COMMENT 'fill all the tables with the datas.' 
  ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe' 
    WITH SERDEPROPERTIES (
  "input.regex" = "([0-9]{10}) ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) (\\S*) (.*)",
  "output.format.string" = "%1$s %2$s %3$s %4$s"
  )
  STORED AS ORC
  LOCATION 's3://path/to/one/of/10/folders/'
  tblproperties ("orc.compress" = "SNAPPY", "skip.header.line.count"="2");
  
  select * from test limit 10;
  </code></pre>
  
  <p>I realize there is probably a very simple solution, but I tried INSERT INTO  in place of CREATE EXTERNAL TABLE, but it understandably complains about the input, and I looked in both the hive and serde documentation for help but was unable to find a reference to adding to an existing store.</p>