title: Java 8 Parallel Streams and Hadoop MapReduce
body: <p><strong>Under what conditions (if any), would it make sense to use Java 8 Parallel Streams in the code for a Hadoop MapReducer reducer?</strong></p>
  
  <p>This question is predicated on the observation that exactly one reducer instance is created for each mapped key, and the assumption that each reducer instance is assigned to exactly one thread.</p>
  
  <p>Thus, it stands to reason that if the set of mapped values for any particular mapped key is sufficiently large, then the runtime of a job could be improved by introducing intra-reducer parallelism (in addition to the inter-reducer parallelism already provided by the framework).</p>
  
  <p>However, I am unsure how other factors would affect this behavior in practice, and so whether or not this conclusion is generalizable enough to be practical, or whether real-world conditions make it too dependent on some other specific conditions to be actually useful (e.g. something like the interaction between the number of other jobs in the queue and the number of processors in the cluster).</p>
  
  <hr>
  
  <p><em>NOTE:  As I am a student, I do not have a readily available cluster to test this on myself.</em></p>