title: Apache Spark - Generate List Of Pairs
body: <p>Given a large file containing data of the form, (V1,V2,...,VN)</p>
  
  <pre><code>2,5
  2,8,9
  2,5,8
  ...
  </code></pre>
  
  <p>I am trying to achieve a list of pairs similar to the following using Spark</p>
  
  <pre><code>((2,5),2)
  ((2,8),2)
  ((2,9),1)
  ((8,9),1)
  ((5,8),1)
  </code></pre>
  
  <p>I tried the suggestions mentioned in response to an <a href="https://stackoverflow.com/questions/11803349/composing-a-list-of-all-pairs">older question</a>, but I have encountered some issues. For example, </p>
  
  <pre><code>val dataRead = sc.textFile(inputFile)
  val itemCounts = dataRead
    .flatMap(line =&gt; line.split(","))
    .map(item =&gt; (item, 1))
    .reduceByKey((a, b) =&gt; a + b)
    .cache()
  val nums = itemCounts.keys
    .filter({case (a) =&gt; a.length &gt; 0})
    .map(x =&gt; x.trim.toInt)
  val pairs = nums.flatMap(x =&gt; nums2.map(y =&gt; (x,y)))
  </code></pre>
  
  <p>I got the error,</p>
  
  <pre><code>scala&gt; val pairs = nums.flatMap(x =&gt; nums.map(y =&gt; (x,y)))
  &lt;console&gt;:27: error: type mismatch;
   found   : org.apache.spark.rdd.RDD[(Int, Int)]
   required: TraversableOnce[?]
         val pairs = nums.flatMap(x =&gt; nums.map(y =&gt; (x,y)))
                                               ^
  </code></pre>
  
  <p>Could someone please point me towards what I might be doing incorrectly, or what might be a better way to achieve the same? Many thanks in advance.</p>