title: PySpark DataFrames - way to enumerate without converting to Pandas?
body: <p>I have a very big <strong>pyspark.sql.dataframe.DataFrame</strong> named df. 
  I need some way of enumerating records- thus, being able to access record with certain index. (or select group of records with indexes range)</p>
  
  <p>In pandas, I could make just </p>
  
  <pre><code>indexes=[2,3,6,7] 
  df[indexes]
  </code></pre>
  
  <p>Here I want something similar, <em>(and without converting dataframe to pandas)</em> </p>
  
  <p>The closest I can get to is:</p>
  
  <ul>
  <li><p>Enumerating all the objects in the original dataframe by:</p>
  
  <pre><code>indexes=np.arange(df.count())
  df_indexed=df.withColumn('index', indexes)
  </code></pre>
  
  <ul>
  <li>Searching for  values I need using where() function. </li>
  </ul></li>
  </ul>
  
  <p>QUESTIONS:</p>
  
  <ol>
  <li>Why it doesn't work and how to make it working? How to add a row to a dataframe?</li>
  <li><p>Would it work later to make something like:</p>
  
  <pre><code> indexes=[2,3,6,7] 
   df1.where("index in indexes").collect()
  </code></pre></li>
  <li><p>Any faster and simpler way to deal with it?</p></li>
  </ol>