title: Scipy Spearman Correlation Coefficient is NaN in Some Cases
body: <p>I'm computing Spearman correlation coefficients for interviewers. It works for Interviewer_1... I don't understand how Scipy interrupts interviewer_2 as having no correlation/0/nan.</p>
  
  <pre><code>import pandas as pd
  from pandas import DataFrame
  import scipy.stats
  
  
  df = pd.DataFrame({'Interviewer': ['Interviewer_1','Interviewer_1','Interviewer_1','Interviewer_1','Interviewer_1','Interviewer_1','Interviewer_1','Interviewer_1','Interviewer_1','Interviewer_1','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2','Interviewer_2'],
                      'Score_1': [-1,-1,-1,1,1,-1,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,1,-1,-1,-1,-1,1,-1],
                      'Score_2': [1,-1,-1,-1,1,1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1]
                      })
  
  df
  </code></pre>
  
  <p>Sample Data Yields:</p>
  
  <pre><code>    Interviewer Score_1 Score_2
  0   Interviewer_1   -1  1
  1   Interviewer_1   -1  -1
  2   Interviewer_1   -1  -1
  3   Interviewer_1   1   -1
  4   Interviewer_1   1   1
  5   Interviewer_1   -1  1
  6   Interviewer_1   -1  -1
  7   Interviewer_1   -1  -1
  8   Interviewer_1   1   -1
  9   Interviewer_1   1   -1
  10  Interviewer_2   -1  -1
  11  Interviewer_2   -1  -1
  12  Interviewer_2   -1  -1
  13  Interviewer_2   -1  -1
  14  Interviewer_2   -1  -1
  15  Interviewer_2   -1  -1
  16  Interviewer_2   -1  -1
  17  Interviewer_2   -1  -1
  18  Interviewer_2   -1  -1
  19  Interviewer_2   -1  -1
  20  Interviewer_2   -1  -1
  21  Interviewer_2   -1  -1
  22  Interviewer_2   -1  -1
  23  Interviewer_2   1   -1
  24  Interviewer_2   -1  -1
  25  Interviewer_2   -1  -1
  26  Interviewer_2   -1  -1
  27  Interviewer_2   -1  -1
  28  Interviewer_2   1   -1
  29  Interviewer_2   -1  -1
  
  df.groupby('Interviewer').sum()
  </code></pre>
  
  <p>Yields the Sum:</p>
  
  <pre><code>           Score_1  Score_2
  Interviewer     
  Interviewer_1   -2  -4
  Interviewer_2   -16 -20
  </code></pre>
  
  <p>Using Scipy:</p>
  
  <pre><code>def applyspearman(row):
      row['Cor'] = scipy.stats.spearmanr(row['Score_1'], row['Score_2'])[0]
      return row
  
  df = df.groupby('Interviewer').apply(applyspearman)
  
  df
      Interviewer Score_1 Score_2 Cor
  0   Interviewer_1   -1  1   -0.089087081
  1   Interviewer_1   -1  -1  -0.089087081
  2   Interviewer_1   -1  -1  -0.089087081
  3   Interviewer_1   1   -1  -0.089087081
  4   Interviewer_1   1   1   -0.089087081
  5   Interviewer_1   -1  1   -0.089087081
  6   Interviewer_1   -1  -1  -0.089087081
  7   Interviewer_1   -1  -1  -0.089087081
  8   Interviewer_1   1   -1  -0.089087081
  9   Interviewer_1   1   -1  -0.089087081
  10  Interviewer_2   -1  -1  
  11  Interviewer_2   -1  -1  
  12  Interviewer_2   -1  -1  
  13  Interviewer_2   -1  -1  
  14  Interviewer_2   -1  -1  
  15  Interviewer_2   -1  -1  
  16  Interviewer_2   -1  -1  
  17  Interviewer_2   -1  -1  
  18  Interviewer_2   -1  -1  
  19  Interviewer_2   -1  -1  
  20  Interviewer_2   -1  -1  
  21  Interviewer_2   -1  -1  
  22  Interviewer_2   -1  -1  
  23  Interviewer_2   1   -1  
  24  Interviewer_2   -1  -1  
  25  Interviewer_2   -1  -1  
  26  Interviewer_2   -1  -1  
  27  Interviewer_2   -1  -1  
  28  Interviewer_2   1   -1  
  29  Interviewer_2   -1  -1
  </code></pre>
  
  <p>I tried using this formula by hand in Excel (rank functions, abs difference, d^2, and sum of d^, and got different results for both interviewers: 
  p = 1 - (6 Σ d^2i)/(n(n^2-1))</p>
  
  <p>interviewer_1, p = 0.878788</p>
  
  <p>interviewer_2, p = 0.993985</p>
  
  <p><strong>Questions</strong>:</p>
  
  <ol>
  <li>Why is Interviewer_2 null? Is the NaN issue related to rank ties?</li>
  <li>Why does Scipy's results differ from my results by hand?</li>
  </ol>