title: Spark - java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.api.java.JavaRDDLike
body: <pre><code>public class SparkDemo {
  
      @SuppressWarnings({ &quot;resource&quot; })
      public static void main(String[] args) {
  
           SparkConf conf = new SparkConf().setAppName(&quot;Spark APP&quot;).setMaster(&quot;spark://xxx.xxx.xxx.xx:7077&quot;);
  
          
          JavaSparkContext sc = new JavaSparkContext(conf);
  
          JavaRDD&lt;String&gt; lines = sc.textFile(&quot;/Users/mchaurasia/file.txt&quot;);
  
          JavaRDD&lt;String&gt; words = lines.flatMap((String s) -&gt; {
              return Arrays.asList(s.split(&quot; &quot;));
          });
  
          JavaPairRDD&lt;String, Integer&gt; pairs = words.mapToPair((String s) -&gt; {
              return new Tuple2&lt;String, Integer&gt;(s, 1);
          });
  
          JavaPairRDD&lt;String, Integer&gt; counts = pairs.reduceByKey((a, b) -&gt; a + b);
  
          for (Tuple2&lt;String, Integer&gt; result : counts.collect()) {
              System.out.println(result._1 + &quot; , &quot; + result._2);
          }
  
          sc.close();
          System.out.println(&quot;================= DONE ===================&quot;);
      }
  </code></pre>
  <p><strong>The above code is working fine when I set master to local(setMaster(&quot;local&quot;)), when I use setMaster(&quot;spark://xxx.xxx.xxx.xx:7077&quot;),</strong>
  I am getting following error --</p>
  <pre><code>6/06/21 09:28:59 WARN TaskSetManager: Lost task 1.0 in stage 0.0 (TID 1, xxx.xxx.xxx.xx): java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.f$3 of type org.apache.spark.api.java.function.FlatMapFunction in instance of org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1
      at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2089)
      at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1261)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1999)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
      at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:483)
      at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
  
  
  Exception in thread &quot;main&quot; org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 5, xxx.xxx.xxx.xx): java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1.f$3 of type org.apache.spark.api.java.function.FlatMapFunction in instance of org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$1$1
  at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2089)
  at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1261)
  at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1999)
  at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
  </code></pre>