title: "java.io.IOException: Connection reset by peer" thrown on the resource manager when launching Spark on Yarn
body: <p>I am trying to launch a Spark job using yarn-client mode on a cluster. I have already tried spark-shell with yarn and I can launch the application. But, I also would like to be able run the driver program from, say eclipse, while using the cluster to run the tasks. I have also added spark-assembly jar to HDFS and point to it by adding (HADOOP_CONF_DIR env variable) to eclipse, although I'm not sure if that's the best way to go about this. </p>
  
  <p>My application does launch on the cluster (as I can see it in the resource manager's monitor) it finishes "successfully" but without any results coming back to the driver. I see the following exception in eclipse console: </p>
  
  <pre><code>WARN  10:11:08,375  Logging.scala:71 -- Lost task 0.0 in stage 1.0 (TID 1, vanpghdcn2.pgdev.sap.corp): java.lang.NullPointerException 
          at org.apache.spark.sql.execution.RDDConversions$$anonfun$rowToRowRdd$1.apply(ExistingRDD.scala:56) 
          at org.apache.spark.sql.execution.RDDConversions$$anonfun$rowToRowRdd$1.apply(ExistingRDD.scala:55) 
          at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$17.apply(RDD.scala:686) 
          at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$17.apply(RDD.scala:686) 
          at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 
          at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) 
          at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) 
          at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 
          at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) 
          at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) 
          at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 
          at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) 
          at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) 
          at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35) 
          at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277) 
          at org.apache.spark.rdd.RDD.iterator(RDD.scala:244) 
          at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:70) 
          at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41) 
          at org.apache.spark.scheduler.Task.run(Task.scala:70) 
          at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213) 
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
          at java.lang.Thread.run(Thread.java:745) 
  
  ERROR 10:11:08,522  Logging.scala:75 -- Task 0 in stage 1.0 failed 4 times; aborting job 
  INFO  10:11:08,538  SparkUtils.scala:67 --           SparkContext stopped 
  </code></pre>
  
  <p>And I get the following in the ResourceManager log: </p>
  
  <pre><code>2015-10-22 10:10:42,603 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: assignedContainer application attempt=appattempt_1445462013958_0012_000001 container=Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: &lt;memory:1024, vCores:1&gt;, Priority: 0, Token: null, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:0, vCores:0&gt;, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 clusterResource=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:10:42,603 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:1024, vCores:1&gt;, usedCapacity=0.041666668, absoluteUsedCapacity=0.041666668, numApps=1, numContainers=1 
  2015-10-22 10:10:42,603 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: assignedContainer queue=root usedCapacity=0.041666668 absoluteUsedCapacity=0.041666668 used=&lt;memory:1024, vCores:1&gt; cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:10:42,604 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : vanpghdcn3.pgdev.sap.corp:41419 for container : container_1445462013958_0012_01_000001 
  2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000001 Container Transitioned from ALLOCATED to ACQUIRED 
  2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Clear node set for appattempt_1445462013958_0012_000001 
  2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: Storing attempt: AppId: application_1445462013958_0012 AttemptId: appattempt_1445462013958_0012_000001 MasterContainer: Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: &lt;memory:1024, vCores:1&gt;, Priority: 0, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] 
  2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from SCHEDULED to ALLOCATED_SAVING 
  2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from ALLOCATED_SAVING to ALLOCATED 
  2015-10-22 10:10:42,606 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Launching masterappattempt_1445462013958_0012_000001 
  2015-10-22 10:10:42,608 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Setting up container Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: &lt;memory:1024, vCores:1&gt;, Priority: 0, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] for AM appattempt_1445462013958_0012_000001 
  2015-10-22 10:10:42,608 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Command to launch container container_1445462013958_0012_01_000001 : {{JAVA_HOME}}/bin/java,-server,-Xmx512m,-Djava.io.tmpdir={{PWD}}/tmp,'-Dspark.eventLog.dir=','-Dspark.driver.port=57819','-Dspark.app.name=Sparca Application','-Dspark.executor.memory=1g','-Dspark.master=yarn-client','-Dspark.executor.id=driver','-Dspark.externalBlockStore.folderName=spark-10391661-8d35-40d9-8242-fe79bdc19d2d','-Dspark.fileserver.uri=&lt;a href="http://10.161.43.118:57820','-Dspark.driver.appUIAddress=http://10.161.43.118:4040','-Dspark.driver.host=10.161.43.118','-Dspark.eventLog.enabled=false','-Dspark.yarn.jar=hdfs://vanpghdcn1.pgdev.sap.corp:8020/data/spark-assembly-1.4.0-hadoop2.6.0.jar','-Dspark.cores.max=6',-Dspark.yarn.app.container.log.dir="&gt;http://10.161.43.118:57820','-Dspark.driver.appUIAddress=http://10.161.43.118:4040','-Dspark.driver.host=10.161.43.118','-Dspark.eventLog.enabled=false','-Dspark.yarn.jar=hdfs://vanpghdcn1.pgdev.sap.corp:8020/data/spark-assembly-1.4.0-hadoop2.6.0.jar','-Dspark.cores.max=6',-Dspark.yarn.app.container.log.dir=&lt;LOG_DIR&gt;,org.apache.spark.deploy.yarn.ExecutorLauncher,--arg,'10.161.43.118:57819',--executor-memory,1024m,--executor-cores,1,--num-executors ,2,1&gt;,&lt;LOG_DIR&gt;/stdout,2&gt;,&lt;LOG_DIR&gt;/stderr 
  2015-10-22 10:10:42,608 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: Create AMRMToken for ApplicationAttempt: appattempt_1445462013958_0012_000001 
  2015-10-22 10:10:42,608 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: Creating password for appattempt_1445462013958_0012_000001 
  2015-10-22 10:10:42,640 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Done launching container Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: &lt;memory:1024, vCores:1&gt;, Priority: 0, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] for AM appattempt_1445462013958_0012_000001 
  2015-10-22 10:10:42,640 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from ALLOCATED to LAUNCHED 
  2015-10-22 10:10:43,613 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000001 Container Transitioned from ACQUIRED to RUNNING 
  2015-10-22 10:10:48,176 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1445462013958_0012_000001 (auth:SIMPLE) 
  2015-10-22 10:10:48,188 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: AM registration appattempt_1445462013958_0012_000001 
  2015-10-22 10:10:48,188 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs IP=10.165.28.145    OPERATION=Register App Master   TARGET=ApplicationMasterService RESULT=SUCCESS  APPID=application_1445462013958_0012 APPATTEMPTID=appattempt_1445462013958_0012_000001 
  2015-10-22 10:10:48,188 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from LAUNCHED to RUNNING 
  2015-10-22 10:10:48,188 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1445462013958_0012 State change from ACCEPTED to RUNNING 
  2015-10-22 10:10:48,632 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000002 Container Transitioned from NEW to ALLOCATED 
  2015-10-22 10:10:48,632 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Allocated Container    TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000002 
  2015-10-22 10:10:48,632 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1445462013958_0012_01_000002 of capacity &lt;memory:2048, vCores:1&gt; on host vanpghdcn3.pgdev.sap.corp:41419, which has 2 containers, &lt;memory:3072, vCores:2&gt; used and &lt;memory:5120, vCores:6&gt; available after allocation 
  2015-10-22 10:10:48,632 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: assignedContainer application attempt=appattempt_1445462013958_0012_000001 container=Container: [ContainerId: container_1445462013958_0012_01_000002, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: &lt;memory:2048, vCores:1&gt;, Priority: 1, Token: null, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:1024, vCores:1&gt;, usedCapacity=0.041666668, absoluteUsedCapacity=0.041666668, numApps=1, numContainers=1 clusterResource=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:10:48,633 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:3072, vCores:2&gt;, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 
  2015-10-22 10:10:48,633 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: assignedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used=&lt;memory:3072, vCores:2&gt; cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000003 Container Transitioned from NEW to ALLOCATED 
  2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Allocated Container    TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000003 
  2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1445462013958_0012_01_000003 of capacity &lt;memory:2048, vCores:1&gt; on host vanpghdcn2.pgdev.sap.corp:36064, which has 1 containers, &lt;memory:2048, vCores:1&gt; used and &lt;memory:6144, vCores:7&gt; available after allocation 
  2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: assignedContainer application attempt=appattempt_1445462013958_0012_000001 container=Container: [ContainerId: container_1445462013958_0012_01_000003, NodeId: vanpghdcn2.pgdev.sap.corp:36064, NodeHttpAddress: vanpghdcn2.pgdev.sap.corp:8042, Resource: &lt;memory:2048, vCores:1&gt;, Priority: 1, Token: null, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:3072, vCores:2&gt;, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 clusterResource=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:10:48,819 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting assigned queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:5120, vCores:3&gt;, usedCapacity=0.20833333, absoluteUsedCapacity=0.20833333, numApps=1, numContainers=3 
  2015-10-22 10:10:48,820 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: assignedContainer queue=root usedCapacity=0.20833333 absoluteUsedCapacity=0.20833333 used=&lt;memory:5120, vCores:3&gt; cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:10:53,253 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : vanpghdcn3.pgdev.sap.corp:41419 for container : container_1445462013958_0012_01_000002 
  2015-10-22 10:10:53,255 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000002 Container Transitioned from ALLOCATED to ACQUIRED 
  2015-10-22 10:10:53,256 INFO org.apache.hadoop.yarn.server.resourcemanager.security.NMTokenSecretManagerInRM: Sending NMToken for nodeId : vanpghdcn2.pgdev.sap.corp:36064 for container : container_1445462013958_0012_01_000003 
  2015-10-22 10:10:53,257 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000003 Container Transitioned from ALLOCATED to ACQUIRED 
  2015-10-22 10:10:53,643 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000002 Container Transitioned from ACQUIRED to RUNNING 
  2015-10-22 10:10:53,830 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000003 Container Transitioned from ACQUIRED to RUNNING 
  2015-10-22 10:10:58,282 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo: checking for deactivate of application :application_1445462013958_0012 
  2015-10-22 10:11:08,349 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: Updating application attempt appattempt_1445462013958_0012_000001 with final state: FINISHING, and exit status: -1000 
  2015-10-22 10:11:08,349 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from RUNNING to FINAL_SAVING 
  2015-10-22 10:11:08,349 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Updating application application_1445462013958_0012 with final state: FINISHING 
  2015-10-22 10:11:08,349 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1445462013958_0012 State change from RUNNING to FINAL_SAVING 
  2015-10-22 10:11:08,350 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Updating info for app: application_1445462013958_0012 
  2015-10-22 10:11:08,350 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from FINAL_SAVING to FINISHING 
  2015-10-22 10:11:08,350 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1445462013958_0012 State change from FINAL_SAVING to FINISHING 
  2015-10-22 10:11:08,453 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: application_1445462013958_0012 unregistered successfully. 
  2015-10-22 10:11:08,692 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000002 Container Transitioned from RUNNING to COMPLETED 
  2015-10-22 10:11:08,692 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp: Completed container: container_1445462013958_0012_01_000002 in state: COMPLETED event:FINISHED 
  2015-10-22 10:11:08,692 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000002 
  2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Released container container_1445462013958_0012_01_000002 of capacity &lt;memory:2048, vCores:1&gt; on host vanpghdcn3.pgdev.sap.corp:41419, which currently has 1 containers, &lt;memory:1024, vCores:1&gt; used and &lt;memory:7168, vCores:7&gt; available, release resources=true 
  2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: default used=&lt;memory:3072, vCores:2&gt; numContainers=2 user=hdfs user-resources=&lt;memory:3072, vCores:2&gt;
  2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: completedContainer container=Container: [ContainerId: container_1445462013958_0012_01_000002, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: &lt;memory:2048, vCores:1&gt;, Priority: 1, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:3072, vCores:2&gt;, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: completedContainer queue=root usedCapacity=0.125 absoluteUsedCapacity=0.125 used=&lt;memory:3072, vCores:2&gt; cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting completed queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:3072, vCores:2&gt;, usedCapacity=0.125, absoluteUsedCapacity=0.125, numApps=1, numContainers=2 
  2015-10-22 10:11:08,693 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application attempt appattempt_1445462013958_0012_000001 released container container_1445462013958_0012_01_000002 on node: host: vanpghdcn3.pgdev.sap.corp:41419 #containers=1 available=&lt;memory:7168, vCores:7&gt; used=&lt;memory:1024, vCores:1&gt; with event: FINISHED 
  2015-10-22 10:11:08,704 INFO org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8050: readAndProcess from client 10.161.43.118 threw exception [java.io.IOException: Connection reset by peer] 
  java.io.IOException: Connection reset by peer 
          at sun.nio.ch.FileDispatcherImpl.read0(Native Method) 
          at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) 
          at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) 
          at sun.nio.ch.IOUtil.read(IOUtil.java:197) 
          at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) 
          at org.apache.hadoop.ipc.Server.channelRead(Server.java:2603) 
          at org.apache.hadoop.ipc.Server.access$2800(Server.java:136) 
          at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1481) 
          at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:771) 
          at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:637) 
          at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:608) 
  2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000003 Container Transitioned from RUNNING to COMPLETED 
  2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp: Completed container: container_1445462013958_0012_01_000003 in state: COMPLETED event:FINISHED 
  2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000003 
  2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Released container container_1445462013958_0012_01_000003 of capacity &lt;memory:2048, vCores:1&gt; on host vanpghdcn2.pgdev.sap.corp:36064, which currently has 0 containers, &lt;memory:0, vCores:0&gt; used and &lt;memory:8192, vCores:8&gt; available, release resources=true 
  2015-10-22 10:11:08,920 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: default used=&lt;memory:1024, vCores:1&gt; numContainers=1 user=hdfs user-resources=&lt;memory:1024, vCores:1&gt;
  2015-10-22 10:11:08,921 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: completedContainer container=Container: [ContainerId: container_1445462013958_0012_01_000003, NodeId: vanpghdcn2.pgdev.sap.corp:36064, NodeHttpAddress: vanpghdcn2.pgdev.sap.corp:8042, Resource: &lt;memory:2048, vCores:1&gt;, Priority: 1, Token: Token { kind: ContainerToken, service: 10.165.28.143:36064 }, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:1024, vCores:1&gt;, usedCapacity=0.041666668, absoluteUsedCapacity=0.041666668, numApps=1, numContainers=1 cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:11:08,921 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: completedContainer queue=root usedCapacity=0.041666668 absoluteUsedCapacity=0.041666668 used=&lt;memory:1024, vCores:1&gt; cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:11:08,921 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting completed queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:1024, vCores:1&gt;, usedCapacity=0.041666668, absoluteUsedCapacity=0.041666668, numApps=1, numContainers=1 
  2015-10-22 10:11:08,921 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application attempt appattempt_1445462013958_0012_000001 released container container_1445462013958_0012_01_000003 on node: host: vanpghdcn2.pgdev.sap.corp:36064 #containers=0 available=&lt;memory:8192, vCores:8&gt; used=&lt;memory:0, vCores:0&gt; with event: FINISHED 
  2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl: container_1445462013958_0012_01_000001 Container Transitioned from RUNNING to COMPLETED 
  2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: Unregistering app attempt : appattempt_1445462013958_0012_000001 
  2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp: Completed container: container_1445462013958_0012_01_000001 in state: COMPLETED event:FINISHED 
  2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=AM Released Container TARGET=SchedulerApp RESULT=SUCCESS  APPID=application_1445462013958_0012 CONTAINERID=container_1445462013958_0012_01_000001 
  2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Released container container_1445462013958_0012_01_000001 of capacity &lt;memory:1024, vCores:1&gt; on host vanpghdcn3.pgdev.sap.corp:41419, which currently has 0 containers, &lt;memory:0, vCores:0&gt; used and &lt;memory:8192, vCores:8&gt; available, release resources=true 
  2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: default used=&lt;memory:0, vCores:0&gt; numContainers=0 user=hdfs user-resources=&lt;memory:0, vCores:0&gt;
  2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: completedContainer container=Container: [ContainerId: container_1445462013958_0012_01_000001, NodeId: vanpghdcn3.pgdev.sap.corp:41419, NodeHttpAddress: vanpghdcn3.pgdev.sap.corp:8042, Resource: &lt;memory:1024, vCores:1&gt;, Priority: 0, Token: Token { kind: ContainerToken, service: 10.165.28.145:41419 }, ] queue=default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:0, vCores:0&gt;, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: completedContainer queue=root usedCapacity=0.0 absoluteUsedCapacity=0.0 used=&lt;memory:0, vCores:0&gt; cluster=&lt;memory:24576, vCores:24&gt;
  2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Re-sorting completed queue: root.default stats: default: capacity=1.0, absoluteCapacity=1.0, usedResources=&lt;memory:0, vCores:0&gt;, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=1, numContainers=0 
  2015-10-22 10:11:09,695 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application attempt appattempt_1445462013958_0012_000001 released container container_1445462013958_0012_01_000001 on node: host: vanpghdcn3.pgdev.sap.corp:41419 #containers=0 available=&lt;memory:8192, vCores:8&gt; used=&lt;memory:0, vCores:0&gt; with event: FINISHED 
  2015-10-22 10:11:09,694 INFO org.apache.hadoop.yarn.server.resourcemanager.security.AMRMTokenSecretManager: Application finished, removing password for appattempt_1445462013958_0012_000001 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1445462013958_0012_000001 State change from FINISHING to FINISHED 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1445462013958_0012 State change from FINISHING to FINISHED 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Application Attempt appattempt_1445462013958_0012_000001 is done. finalState=FINISHED 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hdfs OPERATION=Application Finished - Succeeded  TARGET=RMAppManager RESULT=SUCCESS  APPID=application_1445462013958_0012 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo: Application application_1445462013958_0012 requests cleared 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue: Application removed - appId: application_1445462013958_0012 user: hdfs queue: default #user-pending-applications: 0 #user-active-applications: 0 #queue-pending-applications: 0 #queue-active-applications: 0 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Application removed - appId: application_1445462013958_0012 user: hdfs leaf-queue of parent: root #applications: 0 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary: appId=application_1445462013958_0012,name=Sparca Application,user=hdfs,queue=default,state=FINISHED,trackingUrl=http://vanpghdcn1:8088/proxy/application_1445462013958_0012/,appMasterHost=10.165.28.145,startTime=1445533842429,finishTime=1445533868349,finalStatus=SUCCEEDED,memorySeconds=109990,vcoreSeconds=67,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=&lt;memory:0\, vCores:0&gt;,applicationType=SPARK 
  2015-10-22 10:11:09,696 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Cleaning master appattempt_1445462013958_0012_000001 
  2015-10-22 10:11:10,719 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Null container completed... 
  2015-10-22 10:11:10,925 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Null container completed... 
  </code></pre>
  
  <p>It's worth mentioning that 10.161.43.118 is the machine that I'm running my eclipse on. And my test app is just reading a csv into a dataframe and doing a count.</p>