title: Splitting a specific PySpark df column and create another DF
body: <p>I have a <em>dataframe</em>(say, ac_df) which have 32 different columns. I want to get a specific column and split the values into a chunk of 3 as a single new value and create another df out of it. </p>
  
  <p><code>ac_df['payment_history_1']</code> gives below result</p>
  
  <p><a href="https://i.stack.imgur.com/QN1UY.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/QN1UY.png" alt="enter image description here"></a></p>
  
  <p>I want a new df which is of below structure.</p>
  
  <p><a href="https://i.stack.imgur.com/2XfLr.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/2XfLr.png" alt="enter image description here"></a></p>
  
  <p>For ex: If I take the first row, '000000000000', it is grouped as </p>
  
  <p>'000', '000', '000', '000'</p>
  
  <p>and this creates the first row of new df.</p>
  
  <p>Python equivalent code to perform this task is as below:</p>
  
  <pre><code>temp1 = ac_df['payment_history_1'].str.split(r'(...)', expand=True)
  </code></pre>
  
  <p>In spark, I tried something below:</p>
  
  <pre><code>temp1 = ac_df.select(ac_df['payment_history_1']).rdd.map(lambda each_row: str(each_row[0])).map(lambda y: y.split(r'(...)')).collect()
  </code></pre>
  
  <p>Output:</p>
  
  <pre><code> [['000000000000'], ['000000000003000000000'], ['000000000003000000000000000']]
  </code></pre>
  
  <p>However, I'm unable to move ahead and get the desired result. Can someone suggest?</p>