title: How to compute Spearman correlation in Tensorflow
body: <h1>Problem</h1>
  
  <p>I need to compute the Pearson and Spearman correlations, and use it as metrics in tensorflow.</p>
  
  <p>For Pearson, it's trivial :</p>
  
  <pre><code>tf.contrib.metrics.streaming_pearson_correlation(y_pred, y_true)
  </code></pre>
  
  <p>But for Spearman, I am clueless !</p>
  
  <h1>What I tried :</h1>
  
  <p>From <a href="https://stackoverflow.com/a/42730743/9494790">this answer</a> :</p>
  
  <pre><code>    samples = 1
      predictions_rank = tf.nn.top_k(y_pred, k=samples, sorted=True, name='prediction_rank').indices
      real_rank = tf.nn.top_k(y_true, k=samples, sorted=True, name='real_rank').indices
      rank_diffs = predictions_rank - real_rank
      rank_diffs_squared_sum = tf.reduce_sum(rank_diffs * rank_diffs)
      six = tf.constant(6)
      one = tf.constant(1.0)
      numerator = tf.cast(six * rank_diffs_squared_sum, dtype=tf.float32)
      divider = tf.cast(samples * samples * samples - samples, dtype=tf.float32)
      spearman_batch = one - numerator / divider
  </code></pre>
  
  <p>But this return <code>NaN</code>...</p>
  
  <hr>
  
  <p>Following the <a href="https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient#Definition_and_calculation" rel="noreferrer">definition of Wikipedia</a> :
   <a href="https://i.stack.imgur.com/M8RQM.png" rel="noreferrer"><img src="https://i.stack.imgur.com/M8RQM.png" alt="enter image description here"></a></p>
  
  <p>I tried : </p>
  
  <pre><code>size = tf.size(y_pred)
  indice_of_ranks_pred = tf.nn.top_k(y_pred, k=size)[1]
  indice_of_ranks_label = tf.nn.top_k(y_true, k=size)[1]
  rank_pred = tf.nn.top_k(-indice_of_ranks_pred, k=size)[1]
  rank_label = tf.nn.top_k(-indice_of_ranks_label, k=size)[1]
  rank_pred = tf.to_float(rank_pred)
  rank_label = tf.to_float(rank_label)
  spearman = tf.contrib.metrics.streaming_pearson_correlation(rank_pred, rank_label)
  </code></pre>
  
  <p>But running this I got the following error :</p>
  
  <blockquote>
    <p>tensorflow.python.framework.errors_impl.InvalidArgumentError: input
    must have at least k columns. Had 1, needed 32</p>
    
    <p>[[{{node metrics/spearman/TopKV2}} = TopKV2[T=DT_FLOAT, sorted=true,
    _device="/job:localhost/replica:0/task:0/device:CPU:0"](lambda_1/add, metrics/pearson/pearson_r/variance_predictions/Size)]]</p>
  </blockquote>