Order SparseVectors by the closest distance to given SparseVector I have a Spark dataset containing a column of SparseVector types. Additionally, I have another SparseVector $X$ which is not a part of the dataset. I want to order my dataset according to the closest distance (or similarity) relative to $X$.
Can anyone help me with how to implement this?
it appears that problem was solved with BucketedRandomProjectionLSH. After fit &amp;&amp; transformation, and approxNearestNeighbors resulted dataset contains distCol, which (as per LSH.scala):


  @param distCol Output column for storing the distance between each result row and the key.