title: Map key, value pair based on similarity of their value in Spark
body: <p>I have been learning Spark for several weeks, currently I am trying to group several items or people based on their connection using Spark and Hadoop in Scala. For example, I want to see how football players are connected based on their club history. My "players" rdd would be:</p>
  
  <pre><code>(John, FC Sion)
  (Mike, FC Sion)
  (Bobby, PSV Eindhoven)
  (Hans, FC Sion)
  </code></pre>
  
  <p>I want to have rdd like this:</p>
  
  <pre><code>(John, &lt;Mike, Hans&gt;)
  (Mike, &lt;John, Hans&gt;)
  (Bobby, &lt;&gt;)
  (Hans, &lt;Mike, John&gt;)
  </code></pre>
  
  <p>I plan to use map to accomplish this.</p>
  
  <pre><code>val splitClubs = players.map(player=&gt; (player._1, parseTeammates(player._2, players)))
  </code></pre>
  
  <p>Where parseTeammates is a function that will find players that are also playing for same club (player._2)</p>
  
  <pre><code>// RDD is not a type, how can I insert rdd into a function?
  def parseTeammates(club: String, rdd: RDD) : List[String] = {
      // will generate a list of players that contains same "club" value
      val playerList = rdd.filter(_._1 == club)
      return playerList.values;
  }
  </code></pre>
  
  <p>I get compilation error, type mismatch since the function is expected to return List[String] but instead playerList.values returns org.apache.spark.rdd.RDD[List[String]]. Can anybody help me to get the value of an RDD in its simple form (in my case, List[String]) ?</p>
  
  <p>Also, I think there is a more elegant way to solve this problem, rather than creating a separate RDD and then find a certain key in the new RDD and then returning the value as a list</p>