title: Avoid creation of _$folder$ keys in S3 with hadoop (EMR)
body: <p>I am using an EMR Activity in AWS data pipeline. This EMR Activity is running a hive script in EMR Cluster. It takes dynamo DB as input and stores data in S3. </p>
  
  <p>This is the EMR step used in EMR Activity </p>
  
  <pre><code>s3://elasticmapreduce/libs/script-runner/script-runner.jar,s3://elasticmapreduce/libs/hive/hive-script,--run-hive-script,--hive-versions,latest,--args,-f,s3://my-s3-bucket/hive/my_hive_script.q,-d,DYNAMODB_INPUT_TABLE1=MyTable,-d,S3_OUTPUT_BUCKET=#{output.directoryPath}
  </code></pre>
  
  <p>where </p>
  
  <p>out.direcoryPath is : </p>
  
  <pre><code>s3://my-s3-bucket/output/#{format(@scheduledStartTime,"YYYY-MM-dd")}
  </code></pre>
  
  <p>So this creates one folder and one file in S3. (technically speaking it creates two keys <code>2017-03-18/&lt;some_random_number&gt;</code> and <code>2017-03-18_$folder$</code>)</p>
  
  <pre><code>2017-03-18
  2017-03-18_$folder$
  </code></pre>
  
  <p>How to avoid creation of these extra empty <code>_$folder$</code> files.  </p>
  
  <p>EDIT:
  I found a solution listed at <a href="https://issues.apache.org/jira/browse/HADOOP-10400" rel="noreferrer">https://issues.apache.org/jira/browse/HADOOP-10400</a> but I don't know how to implement it in AWS data pipeline.  </p>