title: Why does Spark's dataset reduce gives different values when using a class?
body: <p>When I use this as my reduce function</p>
  <pre><code>public Integer addCount(Integer integer1, Integer integer2) {
          Integer ret =integer1+integer2;
          return ret;
      }
  </code></pre>
  <p>the reduce function given below returns the correct value</p>
  <pre><code>Integer mapReduce1 = df1.reduce((ReduceFunction&lt;Integer&gt;) mapReduce::addCount);
  </code></pre>
  <p>But when I try to use reduce using an object of My class like below:</p>
  <pre><code> public MapReducer addCount(MapReducer mapReduce1, MapReducer mapReduce2) {
          MapReducer ret = new MapReducer(mapReduce1.count+ mapReduce2.count);
          return ret;
  </code></pre>
  <p>The reduce function returns <code>correct_value + 5</code></p>
  <pre><code>MapReducer mapReduce1 = df.reduce((ReduceFunction&lt;MapReducer&gt;) mapReduce::addCount);
  </code></pre>
  <p>Here MapReducer is a simple class with only one <code>public</code> attribute <code>count</code> and it's getter and setter</p>
  <p>What is going on here? How can I apply rece on a class object properly</p>
  <p>MapReducer class:</p>
  <pre><code>package algorithm;
  
  import org.apache.spark.sql.Row;
  
  import java.io.Serializable;
  
  public class MapReducer implements Serializable {
      public int count;
  
      public int getCount() {
          return count;
      }
  
      public void setCount(int count) {
          this.count = count;
      }
  
      public MapReducer incrementCount(Row row){
          this.count++;
          return this;
      }
      public Integer addCount(Integer integer1, Integer integer2) {
          return integer1+integer2;
      }
  
      public MapReducer addCount(MapReducer sample1, MapReducer sample2) {
  
          MapReducer ret = new MapReducer();
          ret.count = sample1.count+ sample2.count;
  
          return ret;
      }
  
  }
  
  </code></pre>
  <p>Also, I used the following to create the <code>Dataset</code></p>
  <pre><code>MapReducer sampleMR = new MapReducer();
          Dataset&lt;MapReducer&gt; df = InitialTable.quartetsTable
                  .toDF()
                  .map
                          (
                                  (MapFunction&lt;Row, MapReducer&gt;) sampleMR::incrementCount,
                                  Encoders.bean(MapReducer.class)
                          );
  </code></pre>
  <p>I was trying to find out the number of rows on each partition of <code>InitialTable.quartetsTable</code> which is another <code>Dataset</code></p>