title: Apache Spark - reducebyKey - Java -
body: <p>I am trying to understand the working of the <code>reduceByKey</code> in Spark using java as the programming language.</p>
  
  <p>Say I have a sentence "I am who I am".
  I break the sentence into words and store it as a list <code>[I, am, who, I, am]</code>.</p>
  
  <p>Now this function assigns <code>1</code> to each word:</p>
  
  <pre><code>JavaPairRDD&lt;String, Integer&gt; ones = words.mapToPair(new PairFunction&lt;String, String, Integer&gt;() {
      @Override
      public Tuple2&lt;String, Integer&gt; call(String s) {
          return new Tuple2&lt;String, Integer&gt;(s, 1);
      }
  });
  </code></pre>
  
  <p>So the output is something like this: </p>
  
  <pre><code>(I,1) 
  (am,1)
  (who,1)
  (I,1)
  (am,1)
  </code></pre>
  
  <p>Now if I have 3 reducers running, each reducer will get a key and the values associated with that key:</p>
  
  <pre><code>reducer 1:
      (I,1)
      (I,1)
  
  reducer 2:
      (am,1)
      (am,1)
  
  reducer 3:
      (who,1)
  </code></pre>
  
  <p>I wanted to know </p>
  
  <p>a. What exactly happens here in the function below.<br />
  b. What are the parameters <code>new Function2&lt;Integer, Integer, Integer&gt;</code><br />
  c. Basically how the JavaPairRDD is formed.</p>
  
  <pre><code>JavaPairRDD&lt;String, Integer&gt; counts = ones.reduceByKey(new Function2&lt;Integer, Integer, Integer&gt;() {
      @Override
      public Integer call(Integer i1, Integer i2) {
          return i1 + i2;
      }
  });
  </code></pre>