title: Hadoop streaming - remove trailing tab from reducer output
body: <p>I have a hadoop streaming job whose output does not contain key/value pairs. You can think of it as value-only pairs or key-only pairs.</p>
  
  <p>My streaming reducer (a php script) is outputting records separated by newlines. Hadoop streaming treats this as a key with no value, and inserts a tab before the newline. This extra tab is unwanted.</p>
  
  <p>How do I remove it?</p>
  
  <p>I am using hadoop 1.0.3 with AWS EMR. I downloaded the source of hadoop 1.0.3 and found this code in hadoop-1.0.3/src/contrib/streaming/src/java/org/apache/hadoop/streaming/PipeReducer.java :</p>
  
  <pre><code>reduceOutFieldSeparator = job_.get("stream.reduce.output.field.separator", "\t").getBytes("UTF-8");
  </code></pre>
  
  <p>So I tried passing <code>-D stream.reduce.output.field.separator=</code> as an argument to the job with no luck. I also tried <code>-D mapred.textoutputformat.separator=</code> and <code>-D mapreduce.output.textoutputformat.separator=</code> with no luck.</p>
  
  <p>I've searched google of course and nothing I found worked. One search result even stated there was no argument that could be passed to achieve the desired result (though, the hadoop version in that case was really really old).</p>
  
  <p>Here is my code (with added line breaks for readability):</p>
  
  <pre><code>hadoop jar streaming.jar -files s3n://path/to/a/file.json#file.json
      -D mapred.output.compress=true -D stream.reduce.output.field.separator=
      -input s3n://path/to/some/input/*/* -output hdfs:///path/to/output/dir
      -mapper 'php my_mapper.php' -reducer 'php my_reducer.php'
  </code></pre>