title: Convert column values to a sequence in a specific order using Spark Scala
body: <p>I have few products to which I want to assign a Ref_id. I am getting these products from a product table like below :</p>
  <pre><code>val products=ProductDF.select(&quot;key&quot;).distinct()
  </code></pre>
  <p>If I receive any below product order should be as given in the table :</p>
  <p><a href="https://i.stack.imgur.com/k9hj2.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/k9hj2.png" alt="enter image description here" /></a></p>
  <pre><code>val seqDF = Seq((&quot;Service&quot;,0),(&quot;S-Product&quot;,1),(&quot;Speed Product&quot;,2),(&quot;Mobile Plan Small&quot;,3),(&quot;Small Product&quot;,4),(&quot;Day Product&quot;,5)).toDF(&quot;key&quot;, &quot;ref_id&quot;)
  </code></pre>
  <p>If any other product comes , I should be appending the Ref-id after 4 as per below example:</p>
  <p><a href="https://i.stack.imgur.com/IT43f.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/IT43f.png" alt="enter image description here" /></a></p>
  <pre><code>val seqDF = Seq((&quot;Service&quot;,0),(&quot;S-Product&quot;,1),(&quot;Speed Product&quot;,2),(&quot;Small Product&quot;,3),(&quot;Day Product&quot;,4),(&quot;New Product-1&quot;,5),(&quot;New Product2&quot;,6)).toDF(&quot;key&quot;, &quot;ref_id&quot;)
  </code></pre>
  <p>Else If any product missed from the first five Products , Ref ID should be adjusted as below: for eg : Small Product is deleted here</p>
  <p><a href="https://i.stack.imgur.com/PwLWt.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/PwLWt.png" alt="Key              Ref_id
  Service         0
  S-Product       1
  Speed Product   2
  Day Product     3
  New Product-1   4
  New Product-2   5" /></a></p>
  <pre><code>val seqDF = Seq((&quot;Service&quot;,0),(&quot;S-Product&quot;,1),(&quot;Speed Product&quot;,2),(&quot;Day Product&quot;,3),(&quot;New Product-1&quot;,4),(&quot;New Product-2&quot;,5)).toDF(&quot;key&quot;, &quot;ref_id&quot;)
  </code></pre>
  <p>My code was like this for first 5 products , Let me know how to best handle additions and deletions as per above example and assign a Ref_id:</p>
  <p>val seqDF = Seq((&quot;Service&quot;,0),(&quot;S-Product&quot;,1),(&quot;Speed Product&quot;,2),(&quot;Small Product&quot;,3),(&quot;Day Product&quot;,4)).toDF(&quot;key&quot;, &quot;ref_id&quot;)</p>