title: Calculate average using Spark Scala
body: <p>How do I calculate the Average salary per location in Spark Scala with below two data sets ?</p>
  
  <p>File1.csv(Column 4 is salary)  </p>
  
  <pre><code>Ram, 30, Engineer, 40000  
  Bala, 27, Doctor, 30000  
  Hari, 33, Engineer, 50000  
  Siva, 35, Doctor, 60000
  </code></pre>
  
  <p>File2.csv(Column 2 is location)  </p>
  
  <pre><code>Hari, Bangalore  
  Ram, Chennai  
  Bala, Bangalore  
  Siva, Chennai  
  </code></pre>
  
  <p>The above files are not sorted. Need to join these 2 files and find average salary per location. I tried with below code but unable to make it.</p>
  
  <pre><code>val salary = sc.textFile("File1.csv").map(e =&gt; e.split(","))  
  val location = sc.textFile("File2.csv").map(e.split(","))  
  val joined = salary.map(e=&gt;(e(0),e(3))).join(location.map(e=&gt;(e(0),e(1)))  
  val joinedData = joined.sortByKey()  
  val finalData = joinedData.map(v =&gt; (v._1,v._2._1._1,v._2._2))  
  val aggregatedDF = finalData.map(e=&gt; e.groupby(e(2)).agg(avg(e(1))))    
  aggregatedDF.repartition(1).saveAsTextFile("output.txt")  
  </code></pre>
  
  <p>Please help with code and sample output how it will look.</p>
  
  <p>Many Thanks</p>