title: Hadoop: How does OutputCollector work during MapReduce?
body: <p>I want to know if the OutputCollector's 'instance' output used in the map function:
  output.collect(key, value)
  this -output- be storing the key value pairs somewhere?
  even if it emits to the reducer function, their must be an intermediate file, right?
  What are those files? Are they visible and decided by the programer?
  Are the OutputKeyClass, and OutputValueClasses which we specify in the main function these places of storage? [Text.class and IntWritable.class]</p>
  
  <p>Im giving the standard code for Word Count example in MapReduce, which we can find at many places in the net.</p>
  
  <pre><code>public class WordCount {
  
  public static class Map extends MapReduceBase implements Mapper&lt;LongWritable, Text, Text, IntWritable&gt; {
  private final static IntWritable one = new IntWritable(1);
  private Text word = new Text();
  
  public void map(LongWritable key, Text value, OutputCollector&lt;Text, IntWritable&gt; output, Reporter reporter) throws IOException {
  String line = value.toString();
  StringTokenizer tokenizer = new StringTokenizer(line);
  while (tokenizer.hasMoreTokens()) {
  word.set(tokenizer.nextToken());
  output.collect(word, one);
  }
  }
  }
  
  public static class Reduce extends MapReduceBase implements Reducer&lt;Text, IntWritable, Text, IntWritable&gt; {
  public void reduce(Text key, Iterator&lt;IntWritable&gt; values, OutputCollector&lt;Text, IntWritable&gt; output, Reporter reporter) throws IOException {
  int sum = 0;
  while (values.hasNext()) {
  sum += values.next().get();
  }
  output.collect(key, new IntWritable(sum));
  }
  }
  
  public static void main(String[] args) throws Exception {
  JobConf conf = new JobConf(WordCount.class);
  conf.setJobName("wordcount");
  
  conf.setOutputKeyClass(Text.class);
  conf.setOutputValueClass(IntWritable.class);
  
  conf.setMapperClass(Map.class);
  conf.setCombinerClass(Reduce.class);
  conf.setReducerClass(Reduce.class);
  
  conf.setInputFormat(TextInputFormat.class);
  conf.setOutputFormat(TextOutputFormat.class);
  
  FileInputFormat.setInputPaths(conf, new Path(args[0]));
  FileOutputFormat.setOutputPath(conf, new Path(args[1]));    
  JobClient.runJob(conf);
  }
  }
  </code></pre>