title: Creating a new column and rows in dataframe pyspark
body: <p>I have a dataframe like this</p>
  
  <pre><code>id_1    id_desc    cat_1    cat_2
  111      ask        ele     phone
  222      ask hr     ele     phone
  333      ask hr dk  ele     phone
  444      askh       ele     phone
  </code></pre>
  
  <p>If <code>cat_1</code>, <code>cat_2</code> are same for multiple <code>id_1</code>, that association need to be captured as a new column. </p>
  
  <p>Need an output like this,</p>
  
  <pre><code>id_1    id_desc        cat_1    cat_2   id_2
  111      ask             ele    phone   222
  111      ask             ele    phone   333
  111      ask             ele    phone   444
  222      ask hr          ele    phone   111
  222      ask hr          ele    phone   333
  222      ask hr          ele    phone   444
  333      ask hr dk       ele    phone   111
  333      ask hr dk       ele    phone   222
  333      ask hr dk       ele    phone   444
  </code></pre>
  
  <p>how to get this done in python?</p>