title: How to split an RDD into multiple (smaller) RDDs given a max number of rows per RDD, and without using an ID column
body: <p>There are similar questions already asked.
  The most similar is this one:
  <a href="https://stackoverflow.com/questions/26896595/spark-how-to-split-an-rddt-into-seqrddt-and-preserve-the-ordering">Spark: How to split an RDD[T]` into Seq[RDD[T]] and preserve the ordering</a></p>
  
  <p>However, I do not care about preserving the order. Furthermore, I do not have any ID columns in data. What I care about the most is that each row of data is written into a new RDD only once! For this reason, I cannot use a randomSplit, although I am looking forward to such a simple solution. Traversing the partitioned sparkContext will not work either.</p>
  
  <p>I understand that splitting an RDD into several RDDs makes no sense, since RDDs are already made to be processed across many clusters (thus automatically get split).</p>
  
  <p>However, splitting an RDD is a requirement according to a highly complex business logic I need to implement the spark code with and I cannot have it any other way.</p>
  
  <p>The solution that I have is to select ranges from a big RDD, and then simply put each range into a new RDD. However, this looks like a time-consuming task, and therefore not a good solution.</p>
  
  <p>I would appreciate if anyone could give me a hand with this, and keep it down to a beginner level.</p>
  
  <p>SOLUTION THAT WORKED FOR ME:</p>
  
  <pre><code>val numberOfRows = 10000
  
  indexedRDD = RDD.zipWithIndex
  
  for (FROM &lt;-1 to numOfPartitions){
  val tempRDD = indexedRDD.filter(from=&gt; {from._2&gt;from &amp;&amp; from._2 &lt; from+numberOfRows}).map(from=&gt;from._1)
  }
  </code></pre>