title: Calculate Spearman Correlation on a Spark DataFrame
body: <p>I would like to run a Spearman correlation on data that is currently in a Spark DataFrame.  Currently, only the Pearson correlation calculation is available to operate on columns in a DataFrame.  It appears that I can do a Spearman correlation using Spark's MLlib, but I need to pass two RDD[Double] to the function.  The columns I want to compare are Double according to the current schema.</p>
  
  <p>Is there a way to select the columns I want and make the be an array of Doubles so that I can use the MLlib correlation function to get the Spearman correlation coefficient?</p>