We can access HDFS file system and YARN scheduler In the Apache-Hadoop. But Spark has a higher level of coding. Is it possible to access HDFS and YARN in Apache-Spark too?Thanks
Can we access HDFS file system and YARN scheduler in Apache Spark?
bigdata;apache hadoop
Yes.There are examples on spark official document: https://spark.apache.org/examples.htmlJust put your HDFS file uri in your input file path as below (scala syntax).val file = spark.textFile(hdfs://train_data)