title: Spark not executing tasks
body: <p>I cant get the pyspark to work. I added the necessary paths to the system variable <code>SPARK_HOME</code>. I extracted data from my mongodb database and simply converted the obtained list to dataframe. Then, I want to see the dataframe through <code>show()</code> (the last line of code) which gives the following error. My hadoop version is 2.7, pyspark and local spark both are 2.4.1, python 3.6. Java version is 8. </p>
  
  <pre><code>import os
  import sys
  spark_path = r"C:\Tools\spark-2.4.0-bin-hadoop2.7" # spark installed folder
  os.environ['SPARK_HOME'] = spark_path
  sys.path.insert(0, spark_path + "/bin")
  sys.path.insert(0, spark_path + "/python/pyspark/")
  sys.path.insert(0, spark_path + "/python/lib/pyspark.zip")
  sys.path.insert(0, spark_path + "/python/lib/py4j-0.10.7-src.zip")
  
  import pymongo
  from pyspark import SparkContext
  import pandas as pd
  import pyspark
  from nltk.corpus import stopwords
  import re as re
  from pyspark.ml.feature import CountVectorizer , IDF
  from pyspark.mllib.linalg import Vector, Vectors
  from pyspark.mllib.clustering import LDA, LDAModel
  from pyspark.sql.types import StringType
  
  sc = SparkContext(appName = "app")
  # print(sc.version)
  
  from pyspark.sql import SQLContext
  sqlContext = SQLContext(sc)
  
  myclient = pymongo.MongoClient("mongodb://localhost:27017/")
  mydb = myclient["The_Rival_Insights"]
  mycol = mydb["twitter"]
  
  def getText(keyword):
      myquery = {'keyword': keyword}
      for x in mycol.find(myquery):     #x is a dictionary
          a=x["metadata"]
          return a
  
  text=[]
  metadata = getText("uber")    #list is returned
  for b in range(len(metadata)):
      text.append(str(metadata[b]["text"]))
  data = sqlContext.createDataFrame(text,StringType()).show()
  </code></pre>
  
  <p>The following error occurs:</p>
  
  <pre><code>Setting default log level to "WARN".
  To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
  [Stage 0:&gt;                                                          (0 + 1) / 1]2019-04-07 17:50:08 ERROR Executor:91 - Exception in task 0.0 in stage 0.0 (TID 0)
  java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:210)
      at java.net.SocketInputStream.read(SocketInputStream.java:141)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
      at java.io.DataInputStream.readInt(DataInputStream.java:387)
      at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:578)
      at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:571)
      at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:406)
      at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
      at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:255)
      at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:247)
      at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
      at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
      at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
      at org.apache.spark.scheduler.Task.run(Task.scala:121)
      at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
      at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
  2019-04-07 17:50:08 WARN  TaskSetManager:66 - Lost task 0.0 in stage 0.0 (TID 0, localhost, executor driver): java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:210)
      at java.net.SocketInputStream.read(SocketInputStream.java:141)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
      at java.io.DataInputStream.readInt(DataInputStream.java:387)
      at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:578)
      at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:571)
      at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:406)
      at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
      at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:255)
      at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:247)
      at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
      at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
      at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
      at org.apache.spark.scheduler.Task.run(Task.scala:121)
      at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
      at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
  
  2019-04-07 17:50:08 ERROR TaskSetManager:70 - Task 0 in stage 0.0 failed 1 times; aborting job
  Traceback (most recent call last):
    File "C:/Users/Mujtaba Faizi/Documents/Twitter-Sentiment-Analysis-Using-Spark-Streaming-And-Kafka-master/Analysis/sparkml_testing.py", line 41, in &lt;module&gt;
      data = sqlContext.createDataFrame(text,StringType()).show()
    File "F:\Softwares\Anaconda\lib\site-packages\pyspark\sql\dataframe.py", line 378, in show
      print(self._jdf.showString(n, 20, vertical))
    File "F:\Softwares\Anaconda\lib\site-packages\py4j\java_gateway.py", line 1257, in __call__
      answer, self.gateway_client, self.target_id, self.name)
    File "F:\Softwares\Anaconda\lib\site-packages\pyspark\sql\utils.py", line 63, in deco
      return f(*a, **kw)
    File "F:\Softwares\Anaconda\lib\site-packages\py4j\protocol.py", line 328, in get_return_value
      format(target_id, ".", name), value)
  py4j.protocol.Py4JJavaError: An error occurred while calling o37.showString.
  : org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0, localhost, executor driver): java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:210)
      at java.net.SocketInputStream.read(SocketInputStream.java:141)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
      at java.io.DataInputStream.readInt(DataInputStream.java:387)
      at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:578)
      at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:571)
      at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:406)
      at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
      at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:255)
      at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:247)
      at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
      at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
      at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
      at org.apache.spark.scheduler.Task.run(Task.scala:121)
      at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
      at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
  
  Driver stacktrace:
      at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1887)
      at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1875)
      at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1874)
      at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
      at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
      at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1874)
      at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
      at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
      at scala.Option.foreach(Option.scala:257)
      at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
      at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2108)
      at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2057)
      at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2046)
      at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
      at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:737)
      at org.apache.spark.SparkContext.runJob(SparkContext.scala:2061)
      at org.apache.spark.SparkContext.runJob(SparkContext.scala:2082)
      at org.apache.spark.SparkContext.runJob(SparkContext.scala:2101)
      at org.apache.spark.sql.execution.SparkPlan.executeTake(SparkPlan.scala:365)
      at org.apache.spark.sql.execution.CollectLimitExec.executeCollect(limit.scala:38)
      at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$collectFromPlan(Dataset.scala:3384)
      at org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:2545)
      at org.apache.spark.sql.Dataset$$anonfun$head$1.apply(Dataset.scala:2545)
      at org.apache.spark.sql.Dataset$$anonfun$53.apply(Dataset.scala:3365)
      at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78)
      at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
      at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
      at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3364)
      at org.apache.spark.sql.Dataset.head(Dataset.scala:2545)
      at org.apache.spark.sql.Dataset.take(Dataset.scala:2759)
      at org.apache.spark.sql.Dataset.getRows(Dataset.scala:255)
      at org.apache.spark.sql.Dataset.showString(Dataset.scala:292)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
      at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
      at py4j.Gateway.invoke(Gateway.java:282)
      at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
      at py4j.commands.CallCommand.execute(CallCommand.java:79)
      at py4j.GatewayConnection.run(GatewayConnection.java:238)
      at java.lang.Thread.run(Thread.java:748)
  Caused by: java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:210)
      at java.net.SocketInputStream.read(SocketInputStream.java:141)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
      at java.io.DataInputStream.readInt(DataInputStream.java:387)
      at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:578)
      at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:571)
      at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:406)
      at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
      at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
      at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:255)
      at org.apache.spark.sql.execution.SparkPlan$$anonfun$2.apply(SparkPlan.scala:247)
      at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
      at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsInternal$1$$anonfun$apply$24.apply(RDD.scala:836)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
      at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
      at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
      at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
      at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
      at org.apache.spark.scheduler.Task.run(Task.scala:121)
      at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
      at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      ... 1 more
  
  SUCCESS: The process with PID 152396 (child process of PID 151964) has been terminated.
  SUCCESS: The process with PID 151964 (child process of PID 151992) has been terminated.
  SUCCESS: The process with PID 151992 (child process of PID 151592) has been terminated.
  
  Process finished with exit code 1
  </code></pre>
  
  <p>Also, I get another error when I add the code in the end (whilst removing the <code>show()</code> function):</p>
  
  <pre><code>reviews = data.rdd.map(lambda x : x[0]).filter(lambda x: x is not None)
  StopWords = stopwords.words("english")
  tokens = reviews                                                   \
      .map( lambda document: document.strip().lower())               \
      .map( lambda document: re.split(" ", document))          \
      .map( lambda word: [x for x in word if x.isalpha()])           \
      .map( lambda word: [x for x in word if len(x) &gt; 3] )           \
      .map( lambda word: [x for x in word if x not in StopWords])    \
      .zipWithIndex()
  </code></pre>
  
  <p>Clipped Error Messages:</p>
  
  <pre><code>    Setting default log level to "WARN".
      To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
      [Stage 0:&gt;                                                          (0 + 4) / 4]2019-04-07 19:04:30 ERROR PythonRunner:91 - Python worker exited unexpectedly (crashed)
      org.apache.spark.api.python.PythonException: Traceback (most recent call last):
        File "C:\Tools\spark-2.4.0-bin-hadoop2.7\python\lib\pyspark.zip\pyspark\worker.py", line 267, in main
      Exception: Python in worker has different version 2.7 than that in driver 3.6, PySpark cannot run with different minor versions.Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set.
  
          at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.handlePythonException(PythonRunner.scala:452)
          at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:588)
          at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:571)
          at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:406)
          at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
          at scala.collection.Iterator$class.foreach(Iterator.scala:891)
          at org.apache.spark.InterruptibleIterator.foreach(InterruptibleIterator.scala:28)
          at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
          at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
          at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
          at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
          at org.apache.spark.InterruptibleIterator.to(InterruptibleIterator.scala:28)
          at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
          at org.apache.spark.InterruptibleIterator.toBuffer(InterruptibleIterator.scala:28)
          at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
          at org.apache.spark.InterruptibleIterator.toArray(InterruptibleIterator.scala:28)
          at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:945)
          at org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$13.apply(RDD.scala:945)
          at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
          at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2101)
          at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
          at org.apache.spark.scheduler.Task.run(Task.scala:121)
          at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:402)
          at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
          at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:408)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      Caused by: java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(SocketInputStream.java:210)
          at java.net.SocketInputStream.read(SocketInputStream.java:141)
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
          at java.io.DataInputStream.readInt(DataInputStream.java:387)
          at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:578)
          at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:571)
          at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:406)
          at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37
          at 
  org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1945)
          at org.apache.spark.api.python.BasePythonRunner$WriterThread.run(PythonRunner.scala:194)
      2019-04-07 19:04:30 ERROR Executor:91 - Exception in task 2.0 in stage 0.0 (TID 2)
      java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(SocketInputStream.java:210)
          at java.net.SocketInputStream.read(SocketInputStream.java:141)
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
          at java.io.DataInputStream.readInt(DataInputStream.java:387)
          at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:578)
          at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:571)
          at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:406)
          at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
          at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
          at org.apache.spark.api.python.SerDeUtil$AutoBatchedPickler.hasNext(SerDeUtil.scala:153)
          at scala.collection.Iterator$class.foreach(Iterator.scala:891)
          at org.apache.spark.api.python.SerDeUtil$AutoBatchedPickler.foreach(SerDeUtil.scala:148)
          at org.apache.spark.api.python.PythonRDD$.writeIteratorToStream(PythonRDD.scala:224)
          at org.apache.spark.api.python.PythonRunner$$anon$2.writeIteratorToStream(PythonRunner.scala:557)
          at org.apache.spark.api.python.BasePythonRunner$WriterThread$$anonfun$run$1.apply(PythonRunner.scala:345)
          at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1945)
          at org.apache.spark.api.python.BasePythonRunner$WriterThread.run(PythonRunner.scala:194)
      2019-04-07 19:04:30 WARN  TaskSetManager:66 - Lost task 2.0 in stage 0.0 (TID 2, localhost, executor driver): java.net.SocketException: Connection reset
          at java.net.SocketInputStream.read(SocketInputStream.java:210)
          at java.net.SocketInputStream.read(SocketInputStream.java:141)
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
          at java.io.DataInputStream.readInt(DataInputStream.java:387)
          at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:578)
          at org.apache.spark.api.python.PythonRunner$$anon$1.read(PythonRunner.scala:571)
          at org.apache.spark.api.python.BasePythonRunner$ReaderIterator.hasNext(PythonRunner.scala:406)
          at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
          at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
          at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:409)
          at org.apache.spark.api.python.SerDeUtil$AutoBatchedPickler.hasNext(SerDeUtil.scala:153)
          at scala.collection.Iterator$class.foreach(Iterator.scala:891)
          at org.apache.spark.api.python.SerDeUtil$AutoBatchedPickler.foreach(SerDeUtil.scala:148)
          at org.apache.spark.api.python.PythonRDD$.writeIteratorToStream(PythonRDD.scala:224)
          at org.apache.spark.api.python.PythonRunner$$anon$2.writeIteratorToStream(PythonRunner.scala:557)
          at org.apache.spark.api.python.BasePythonRunner$WriterThread$$anonfun$run$1.apply(PythonRunner.scala:345)
          at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1945)
          at org.apache.spark.api.python.BasePythonRunner$WriterThread.run(PythonRunner.scala:194)
  
      2019-04-07 19:04:30 ERROR TaskSetManager:70 - Task 2 in stage 0.0 failed 1 times; aborting job
      Traceback (most recent call last):
        File "C:/Users/Mujtaba Faizi/Documents/Twitter-Sentiment-Analysis-Using-Spark-Streaming-And-Kafka-master/Analysis/sparkml_testing.py", line 52, in &lt;module&gt;
          .map( lambda word: [x for x in word if x not in StopWords])    \
        File "F:\Softwares\Anaconda\lib\site-packages\pyspark\rdd.py", line 2174, in zipWithIndex
          nums = self.mapPartitions(lambda it: [sum(1 for i in it)]).collect()
        File "F:\Softwares\Anaconda\lib\site-packages\pyspark\rdd.py", line 816, in collect
          sock_info = self.ctx._jvm.PythonRDD.collectAndServe(self._jrdd.rdd())
        File "F:\Softwares\Anaconda\lib\site-packages\py4j\java_gateway.py", line 1257, in __call__
          answer, self.gateway_client, self.target_id, self.name)
        File "F:\Softwares\Anaconda\lib\site-packages\pyspark\sql\utils.py", line 63, in deco
          return f(*a, **kw)
        File "F:\Softwares\Anaconda\lib\site-packages\py4j\protocol.py", line 328, in get_return_value
          format(target_id, ".", name), value)
      py4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
      : org.apache.spark.SparkException: Job aborted due to stage failure: Task 2 in stage 0.0 failed 1 times, most recent failure: Lost task 2.0 in stage 0.0 (TID 2, localhost, executor driver): java.net.SocketException: Connection reset
  </code></pre>