title: sample spark CSV and JSON program not running in windows
body: <p>I am running spark program in Windows 10 machine.</p>
  
  <p>I am trying to run the below spark program</p>
  
  <pre><code>import org.apache.spark.SparkContext
  import org.apache.spark.SparkContext._
  import org.apache.spark.SparkConf
  import org.apache.spark.sql.Column
  import org.apache.spark.sql.functions._
  import org.apache.spark.sql.SQLContext
  import org.apache.spark.sql._
  import org.apache.spark.sql.SQLImplicits
  import org.apache.spark.sql.expressions.Window
  import org.apache.spark.sql.TypedColumn
  import org.apache.spark.sql.Encoder
  import org.apache.spark.sql.Encoders
  import com.databricks.spark.csv
  
  object json1 {
  
    def main(args : Array[String]){
      val conf = new SparkConf().setAppName("Simple Application").setMaster("local[2]").set("spark.executor.memory", "1g")
      val sc = new org.apache.spark.SparkContext(conf)
      val sqlc = new org.apache.spark.sql.SQLContext(sc)
  
      val NyseDF = sqlc.load("com.databricks.spark.csv",Map("path" -&gt; args(0),"header"-&gt;"true"))
      NyseDF.registerTempTable("NYSE")
      NyseDF.printSchema()
  
    }
  
  
  
  }
  </code></pre>
  
  <p>When i run the program through Run application mode in eclispse with passing arguments</p>
  
  <p>as</p>
  
  <p>src/test/resources/demo.text</p>
  
  <p>It fails with below error.</p>
  
  <pre><code>Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
  16/10/10 11:02:18 INFO SparkContext: Running Spark version 1.6.0
  16/10/10 11:02:18 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  16/10/10 11:02:18 INFO SecurityManager: Changing view acls to: subho
  16/10/10 11:02:18 INFO SecurityManager: Changing modify acls to: subho
  16/10/10 11:02:18 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(subho); users with modify permissions: Set(subho)
  16/10/10 11:02:19 INFO Utils: Successfully started service 'sparkDriver' on port 61108.
  16/10/10 11:02:20 INFO Slf4jLogger: Slf4jLogger started
  16/10/10 11:02:20 INFO Remoting: Starting remoting
  16/10/10 11:02:20 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@192.168.1.116:61121]
  16/10/10 11:02:20 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 61121.
  16/10/10 11:02:20 INFO SparkEnv: Registering MapOutputTracker
  16/10/10 11:02:20 INFO SparkEnv: Registering BlockManagerMaster
  16/10/10 11:02:21 INFO DiskBlockManager: Created local directory at C:\Users\subho\AppData\Local\Temp\blockmgr-69afda02-ccd1-41d1-aa25-830ba366a75c
  16/10/10 11:02:21 INFO MemoryStore: MemoryStore started with capacity 1128.4 MB
  16/10/10 11:02:21 INFO SparkEnv: Registering OutputCommitCoordinator
  16/10/10 11:02:21 INFO Utils: Successfully started service 'SparkUI' on port 4040.
  16/10/10 11:02:21 INFO SparkUI: Started SparkUI at http://192.168.1.116:4040
  16/10/10 11:02:21 INFO Executor: Starting executor ID driver on host localhost
  16/10/10 11:02:21 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 61132.
  16/10/10 11:02:21 INFO NettyBlockTransferService: Server created on 61132
  16/10/10 11:02:21 INFO BlockManagerMaster: Trying to register BlockManager
  16/10/10 11:02:21 INFO BlockManagerMasterEndpoint: Registering block manager localhost:61132 with 1128.4 MB RAM, BlockManagerId(driver, localhost, 61132)
  16/10/10 11:02:21 INFO BlockManagerMaster: Registered BlockManager
  16/10/10 11:02:23 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 107.7 KB, free 107.7 KB)
  16/10/10 11:02:23 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 9.8 KB, free 117.5 KB)
  16/10/10 11:02:23 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:61132 (size: 9.8 KB, free: 1128.4 MB)
  16/10/10 11:02:23 INFO SparkContext: Created broadcast 0 from textFile at TextFile.scala:30
  16/10/10 11:02:23 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
  java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
      at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:278)
      at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:300)
      at org.apache.hadoop.util.Shell.&lt;clinit&gt;(Shell.java:293)
      at org.apache.hadoop.util.StringUtils.&lt;clinit&gt;(StringUtils.java:76)
      at org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:362)
      at org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$33.apply(SparkContext.scala:1015)
      at org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$33.apply(SparkContext.scala:1015)
      at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:176)
      at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:176)
      at scala.Option.map(Option.scala:146)
      at org.apache.spark.rdd.HadoopRDD.getJobConf(HadoopRDD.scala:176)
      at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:195)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
      at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
      at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1293)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
      at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
      at org.apache.spark.rdd.RDD.take(RDD.scala:1288)
      at com.databricks.spark.csv.CsvRelation.firstLine$lzycompute(CsvRelation.scala:174)
      at com.databricks.spark.csv.CsvRelation.firstLine(CsvRelation.scala:169)
      at com.databricks.spark.csv.CsvRelation.inferSchema(CsvRelation.scala:147)
      at com.databricks.spark.csv.CsvRelation.&lt;init&gt;(CsvRelation.scala:70)
      at com.databricks.spark.csv.DefaultSource.createRelation(DefaultSource.scala:138)
      at com.databricks.spark.csv.DefaultSource.createRelation(DefaultSource.scala:40)
      at com.databricks.spark.csv.DefaultSource.createRelation(DefaultSource.scala:28)
      at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:158)
      at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119)
      at org.apache.spark.sql.SQLContext.load(SQLContext.scala:1153)
      at json1$.main(json1.scala:22)
      at json1.main(json1.scala)
  Exception in thread "main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/C:/Users/subho/Desktop/code-master/simple-spark-project/src/test/resources/demo.text
      at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:251)
      at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:270)
      at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:199)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
      at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
      at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
      at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1293)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
      at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
      at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
      at org.apache.spark.rdd.RDD.take(RDD.scala:1288)
      at com.databricks.spark.csv.CsvRelation.firstLine$lzycompute(CsvRelation.scala:174)
      at com.databricks.spark.csv.CsvRelation.firstLine(CsvRelation.scala:169)
      at com.databricks.spark.csv.CsvRelation.inferSchema(CsvRelation.scala:147)
      at com.databricks.spark.csv.CsvRelation.&lt;init&gt;(CsvRelation.scala:70)
      at com.databricks.spark.csv.DefaultSource.createRelation(DefaultSource.scala:138)
      at com.databricks.spark.csv.DefaultSource.createRelation(DefaultSource.scala:40)
      at com.databricks.spark.csv.DefaultSource.createRelation(DefaultSource.scala:28)
      at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:158)
      at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119)
      at org.apache.spark.sql.SQLContext.load(SQLContext.scala:1153)
      at json1$.main(json1.scala:22)
      at json1.main(json1.scala)
  16/10/10 11:02:23 INFO SparkContext: Invoking stop() from shutdown hook
  16/10/10 11:02:23 INFO SparkUI: Stopped Spark web UI at http://192.168.1.116:4040
  16/10/10 11:02:23 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
  16/10/10 11:02:23 INFO MemoryStore: MemoryStore cleared
  16/10/10 11:02:23 INFO BlockManager: BlockManager stopped
  16/10/10 11:02:24 INFO BlockManagerMaster: BlockManagerMaster stopped
  16/10/10 11:02:24 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
  16/10/10 11:02:24 INFO SparkContext: Successfully stopped SparkContext
  16/10/10 11:02:24 INFO ShutdownHookManager: Shutdown hook called
  16/10/10 11:02:24 INFO ShutdownHookManager: Deleting directory C:\Users\subho\AppData\Local\Temp\spark-7f53ea20-a38c-46d5-8476-a1ae040736ac
  </code></pre>
  
  <p>Below is the main error msg</p>
  
  <p>Input path does not exist: file:/C:/Users/subho/Desktop/code-master/simple-spark-project/src/test/resources/demo.text</p>
  
  <p>I have the file in the below location.</p>
  
  <p><a href="http://i.stack.imgur.com/ugXF7.jpg" rel="nofollow">!</a>]<a href="http://i.stack.imgur.com/ugXF7.jpg" rel="nofollow">1</a></p>
  
  <p>When i ran the below program it ran sucussfully,</p>
  
  <pre><code>import org.apache.spark.SparkContext
  import org.apache.spark.SparkContext._
  import org.apache.spark.SparkConf
  import org.apache.spark.sql.Column
  import org.apache.spark.sql.functions._
  import org.apache.spark.sql.SQLContext
  import org.apache.spark.sql._
  import org.apache.spark.sql.SQLImplicits
  import org.apache.spark.sql.expressions.Window
  import org.apache.spark.sql.TypedColumn
  import org.apache.spark.sql.Encoder
  import org.apache.spark.sql.Encoders
  import com.databricks.spark.csv
  
  object json1 {
  
    def main(args : Array[String]){
      val conf = new SparkConf().setAppName("Simple Application").setMaster("local[2]").set("spark.executor.memory", "1g")
      val sc = new org.apache.spark.SparkContext(conf)
      val sqlc = new org.apache.spark.sql.SQLContext(sc)
  
     /* val NyseDF = sqlc.load("com.databricks.spark.csv",Map("path" -&gt; args(0),"header"-&gt;"true"))
      NyseDF.registerTempTable("NYSE")
      NyseDF.printSchema()
  
      print(sqlc.sql("select distinct(symbol) from NYSE").collect().toList)*/
  
      val PersonDF = sqlc.jsonFile("src/test/resources/Person.json")
  
     // PersonDF.printSchema()
  
      PersonDF.registerTempTable("Person")
  
      sqlc.sql("select * from Person where age &lt; 60").collect().foreach(print)
  
    }
  </code></pre>
  
  <p>Below is the log file.</p>
  
  <pre><code>Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
  16/10/10 11:54:12 INFO SparkContext: Running Spark version 1.6.0
  16/10/10 11:54:13 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  16/10/10 11:54:13 INFO SecurityManager: Changing view acls to: subho
  16/10/10 11:54:13 INFO SecurityManager: Changing modify acls to: subho
  16/10/10 11:54:13 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(subho); users with modify permissions: Set(subho)
  16/10/10 11:54:14 INFO Utils: Successfully started service 'sparkDriver' on port 51113.
  16/10/10 11:54:14 INFO Slf4jLogger: Slf4jLogger started
  16/10/10 11:54:14 INFO Remoting: Starting remoting
  16/10/10 11:54:15 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriverActorSystem@192.168.1.116:51126]
  16/10/10 11:54:15 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 51126.
  16/10/10 11:54:15 INFO SparkEnv: Registering MapOutputTracker
  16/10/10 11:54:15 INFO SparkEnv: Registering BlockManagerMaster
  16/10/10 11:54:15 INFO DiskBlockManager: Created local directory at C:\Users\subho\AppData\Local\Temp\blockmgr-a52a5d5a-075b-4859-8434-935fdaba8538
  16/10/10 11:54:15 INFO MemoryStore: MemoryStore started with capacity 1128.4 MB
  16/10/10 11:54:15 INFO SparkEnv: Registering OutputCommitCoordinator
  16/10/10 11:54:15 INFO Utils: Successfully started service 'SparkUI' on port 4040.
  16/10/10 11:54:15 INFO SparkUI: Started SparkUI at http://192.168.1.116:4040
  16/10/10 11:54:15 INFO Executor: Starting executor ID driver on host localhost
  16/10/10 11:54:15 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 51137.
  16/10/10 11:54:15 INFO NettyBlockTransferService: Server created on 51137
  16/10/10 11:54:15 INFO BlockManagerMaster: Trying to register BlockManager
  16/10/10 11:54:15 INFO BlockManagerMasterEndpoint: Registering block manager localhost:51137 with 1128.4 MB RAM, BlockManagerId(driver, localhost, 51137)
  16/10/10 11:54:15 INFO BlockManagerMaster: Registered BlockManager
  16/10/10 11:54:17 INFO JSONRelation: Listing file:/C:/Users/subho/Desktop/code-master/simple-spark-project/src/test/resources/Person.json on driver
  16/10/10 11:54:17 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
  java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
      at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:278)
      at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:300)
      at org.apache.hadoop.util.Shell.&lt;clinit&gt;(Shell.java:293)
      at org.apache.hadoop.util.StringUtils.&lt;clinit&gt;(StringUtils.java:76)
      at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.setInputPaths(FileInputFormat.java:447)
      at org.apache.spark.sql.execution.datasources.json.JSONRelation.org$apache$spark$sql$execution$datasources$json$JSONRelation$$createBaseRdd(JSONRelation.scala:98)
      at org.apache.spark.sql.execution.datasources.json.JSONRelation$$anonfun$4$$anonfun$apply$1.apply(JSONRelation.scala:115)
      at org.apache.spark.sql.execution.datasources.json.JSONRelation$$anonfun$4$$anonfun$apply$1.apply(JSONRelation.scala:115)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.sql.execution.datasources.json.JSONRelation$$anonfun$4.apply(JSONRelation.scala:115)
      at org.apache.spark.sql.execution.datasources.json.JSONRelation$$anonfun$4.apply(JSONRelation.scala:109)
      at scala.Option.getOrElse(Option.scala:121)
      at org.apache.spark.sql.execution.datasources.json.JSONRelation.dataSchema$lzycompute(JSONRelation.scala:109)
      at org.apache.spark.sql.execution.datasources.json.JSONRelation.dataSchema(JSONRelation.scala:108)
      at org.apache.spark.sql.sources.HadoopFsRelation.schema$lzycompute(interfaces.scala:636)
      at org.apache.spark.sql.sources.HadoopFsRelation.schema(interfaces.scala:635)
      at org.apache.spark.sql.execution.datasources.LogicalRelation.&lt;init&gt;(LogicalRelation.scala:37)
      at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:125)
      at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:109)
      at org.apache.spark.sql.DataFrameReader.json(DataFrameReader.scala:244)
      at org.apache.spark.sql.SQLContext.jsonFile(SQLContext.scala:1011)
      at json1$.main(json1.scala:28)
      at json1.main(json1.scala)
  16/10/10 11:54:18 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 128.0 KB, free 128.0 KB)
  16/10/10 11:54:18 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 14.1 KB, free 142.1 KB)
  16/10/10 11:54:18 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:51137 (size: 14.1 KB, free: 1128.4 MB)
  16/10/10 11:54:18 INFO SparkContext: Created broadcast 0 from jsonFile at json1.scala:28
  16/10/10 11:54:18 INFO FileInputFormat: Total input paths to process : 1
  16/10/10 11:54:18 INFO SparkContext: Starting job: jsonFile at json1.scala:28
  16/10/10 11:54:18 INFO DAGScheduler: Got job 0 (jsonFile at json1.scala:28) with 2 output partitions
  16/10/10 11:54:18 INFO DAGScheduler: Final stage: ResultStage 0 (jsonFile at json1.scala:28)
  16/10/10 11:54:18 INFO DAGScheduler: Parents of final stage: List()
  16/10/10 11:54:18 INFO DAGScheduler: Missing parents: List()
  16/10/10 11:54:18 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[3] at jsonFile at json1.scala:28), which has no missing parents
  16/10/10 11:54:18 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 4.2 KB, free 146.3 KB)
  16/10/10 11:54:18 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.4 KB, free 148.6 KB)
  16/10/10 11:54:18 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:51137 (size: 2.4 KB, free: 1128.4 MB)
  16/10/10 11:54:18 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1006
  16/10/10 11:54:18 INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 0 (MapPartitionsRDD[3] at jsonFile at json1.scala:28)
  16/10/10 11:54:18 INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks
  16/10/10 11:54:18 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, partition 0,PROCESS_LOCAL, 2113 bytes)
  16/10/10 11:54:18 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, localhost, partition 1,PROCESS_LOCAL, 2113 bytes)
  16/10/10 11:54:18 INFO Executor: Running task 1.0 in stage 0.0 (TID 1)
  16/10/10 11:54:18 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
  16/10/10 11:54:18 INFO HadoopRDD: Input split: file:/C:/Users/subho/Desktop/code-master/simple-spark-project/src/test/resources/Person.json:0+92
  16/10/10 11:54:18 INFO HadoopRDD: Input split: file:/C:/Users/subho/Desktop/code-master/simple-spark-project/src/test/resources/Person.json:92+93
  16/10/10 11:54:18 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
  16/10/10 11:54:18 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id
  16/10/10 11:54:18 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
  16/10/10 11:54:18 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap
  16/10/10 11:54:18 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
  16/10/10 11:54:18 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id
  16/10/10 11:54:19 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 2886 bytes result sent to driver
  16/10/10 11:54:19 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 2886 bytes result sent to driver
  16/10/10 11:54:19 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 1287 ms on localhost (1/2)
  16/10/10 11:54:19 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 1264 ms on localhost (2/2)
  16/10/10 11:54:19 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 
  16/10/10 11:54:19 INFO DAGScheduler: ResultStage 0 (jsonFile at json1.scala:28) finished in 1.314 s
  16/10/10 11:54:19 INFO DAGScheduler: Job 0 finished: jsonFile at json1.scala:28, took 1.413653 s
  16/10/10 11:54:20 INFO BlockManagerInfo: Removed broadcast_1_piece0 on localhost:51137 in memory (size: 2.4 KB, free: 1128.4 MB)
  16/10/10 11:54:20 INFO ContextCleaner: Cleaned accumulator 1
  16/10/10 11:54:20 INFO BlockManagerInfo: Removed broadcast_0_piece0 on localhost:51137 in memory (size: 14.1 KB, free: 1128.4 MB)
  16/10/10 11:54:21 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 59.6 KB, free 59.6 KB)
  16/10/10 11:54:21 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 13.8 KB, free 73.3 KB)
  16/10/10 11:54:21 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on localhost:51137 (size: 13.8 KB, free: 1128.4 MB)
  16/10/10 11:54:21 INFO SparkContext: Created broadcast 2 from collect at json1.scala:34
  16/10/10 11:54:21 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 128.0 KB, free 201.3 KB)
  16/10/10 11:54:21 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 14.1 KB, free 215.4 KB)
  16/10/10 11:54:21 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on localhost:51137 (size: 14.1 KB, free: 1128.3 MB)
  16/10/10 11:54:21 INFO SparkContext: Created broadcast 3 from collect at json1.scala:34
  16/10/10 11:54:21 INFO FileInputFormat: Total input paths to process : 1
  16/10/10 11:54:21 INFO SparkContext: Starting job: collect at json1.scala:34
  16/10/10 11:54:21 INFO DAGScheduler: Got job 1 (collect at json1.scala:34) with 2 output partitions
  16/10/10 11:54:21 INFO DAGScheduler: Final stage: ResultStage 1 (collect at json1.scala:34)
  16/10/10 11:54:21 INFO DAGScheduler: Parents of final stage: List()
  16/10/10 11:54:21 INFO DAGScheduler: Missing parents: List()
  16/10/10 11:54:21 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[9] at collect at json1.scala:34), which has no missing parents
  16/10/10 11:54:21 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 7.6 KB, free 223.0 KB)
  16/10/10 11:54:21 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 4.1 KB, free 227.1 KB)
  16/10/10 11:54:21 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on localhost:51137 (size: 4.1 KB, free: 1128.3 MB)
  16/10/10 11:54:21 INFO SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:1006
  16/10/10 11:54:21 INFO DAGScheduler: Submitting 2 missing tasks from ResultStage 1 (MapPartitionsRDD[9] at collect at json1.scala:34)
  16/10/10 11:54:21 INFO TaskSchedulerImpl: Adding task set 1.0 with 2 tasks
  16/10/10 11:54:21 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 2, localhost, partition 0,PROCESS_LOCAL, 2113 bytes)
  16/10/10 11:54:21 INFO TaskSetManager: Starting task 1.0 in stage 1.0 (TID 3, localhost, partition 1,PROCESS_LOCAL, 2113 bytes)
  16/10/10 11:54:21 INFO Executor: Running task 0.0 in stage 1.0 (TID 2)
  16/10/10 11:54:21 INFO Executor: Running task 1.0 in stage 1.0 (TID 3)
  16/10/10 11:54:21 INFO HadoopRDD: Input split: file:/C:/Users/subho/Desktop/code-master/simple-spark-project/src/test/resources/Person.json:92+93
  16/10/10 11:54:21 INFO HadoopRDD: Input split: file:/C:/Users/subho/Desktop/code-master/simple-spark-project/src/test/resources/Person.json:0+92
  16/10/10 11:54:22 INFO BlockManagerInfo: Removed broadcast_2_piece0 on localhost:51137 in memory (size: 13.8 KB, free: 1128.4 MB)
  16/10/10 11:54:22 INFO GenerateUnsafeProjection: Code generated in 548.352258 ms
  16/10/10 11:54:22 INFO GeneratePredicate: Code generated in 5.245214 ms
  16/10/10 11:54:22 INFO Executor: Finished task 1.0 in stage 1.0 (TID 3). 2283 bytes result sent to driver
  16/10/10 11:54:22 INFO Executor: Finished task 0.0 in stage 1.0 (TID 2). 2536 bytes result sent to driver
  16/10/10 11:54:22 INFO TaskSetManager: Finished task 1.0 in stage 1.0 (TID 3) in 755 ms on localhost (1/2)
  16/10/10 11:54:22 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 2) in 759 ms on localhost (2/2)
  16/10/10 11:54:22 INFO DAGScheduler: ResultStage 1 (collect at json1.scala:34) finished in 0.760 s
  16/10/10 11:54:22 INFO DAGScheduler: Job 1 finished: collect at json1.scala:34, took 0.779652 s
  16/10/10 11:54:22 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 
  [53,Barack,Obama]16/10/10 11:54:22 INFO SparkContext: Invoking stop() from shutdown hook
  16/10/10 11:54:22 INFO SparkUI: Stopped Spark web UI at http://192.168.1.116:4040
  16/10/10 11:54:22 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
  16/10/10 11:54:22 INFO MemoryStore: MemoryStore cleared
  16/10/10 11:54:22 INFO BlockManager: BlockManager stopped
  16/10/10 11:54:22 INFO BlockManagerMaster: BlockManagerMaster stopped
  16/10/10 11:54:22 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
  16/10/10 11:54:22 INFO SparkContext: Successfully stopped SparkContext
  16/10/10 11:54:22 INFO ShutdownHookManager: Shutdown hook called
  16/10/10 11:54:22 INFO ShutdownHookManager: Deleting directory C:\Users\subho\AppData\Local\Temp\spark-6cab6329-83f1-4af4-b64c-c869550405a4
  16/10/10 11:54:22 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
  </code></pre>
  
  <p>Thanks and Regards,</p>