title: Scikit-learn χ² (chi-squared) statistic and corresponding contingency table
body: <p>In the docs for the chi-squared univariate feature selection function of scikit-learn <a href="http://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.chi2.html" rel="nofollow noreferrer">http://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.chi2.html</a>, it states</p>
  
  <blockquote>
    <p>This score can be used to select the n_features features with the highest values for the χ² (chi-square) statistic from X, which must contain booleans or <strong>frequencies</strong> (e.g., term counts in document classification), relative to the classes.</p>
  </blockquote>
  
  <p>I am struggling to understand what the corresponding contingency table would look like, especially in the case of frequency features.</p>
  
  <p>For example, consider the below dataset with boolean features and targets:</p>
  
  <pre><code>import numpy as np
  
  &gt;&gt;&gt; X = np.random.randint(2, size=50).reshape(10, 5)
  array([[1, 0, 0, 0, 1],
         [1, 1, 0, 1, 1],
         [1, 0, 0, 0, 0],
         [0, 0, 0, 0, 0],
         [0, 0, 0, 0, 1],
         [1, 0, 0, 0, 1],
         [1, 0, 1, 1, 1],
         [0, 1, 1, 0, 0],
         [1, 0, 1, 1, 1],
         [1, 1, 1, 1, 0]])
  
  &gt;&gt;&gt; y = np.random.randint(2, size=10)
  array([1, 0, 0, 0, 1, 1, 1, 1, 0, 1])
  </code></pre>
  
  <p>To construct the contingency table with respect to the first feature, we can do this (excuse my PEP8 violation)</p>
  
  <pre><code>import scipy as sp
  
  &gt;&gt;&gt; contingency_table = sp.sparse.coo_matrix(
  ...    (np.ones_like(y), (X[:, 0], y)), 
  ...    shape=(np.unique(X[:, 0]).shape[0], np.unique(y).shape[0])).A
  array([[1, 2],
         [3, 4]])
  </code></pre>
  
  <p>So now I can calculate the chi-squared statistic and its p-values</p>
  
  <pre><code>&gt;&gt;&gt; sp.stats.chi2_contingency(contingency_table)
  (0.17857142857142855,
   0.67260381744151676,
   1,
   array([[ 1.2,  1.8],
         [ 2.8,  4.2]]))
  </code></pre>
  
  <p>And this ought to be consistent with scikit-learn's <code>chi2</code></p>
  
  <pre><code>from sklearn.feature_selection import chi2
  
  &gt;&gt;&gt; chi2_, pval = chi2(X, y)
  &gt;&gt;&gt; chi2_[0], pval[0]
  (0.023809523809523787, 0.87737055606414338)
  </code></pre>
  
  <p>...Nope. Have I misinterpreted something? </p>
  
  <p>Also, what does the contingency table look like in the case of frequencies? I assumed it would be something like</p>
  
  <pre><code>contingency_table = sp.sparse.coo_matrix(
      (np.ones_like(y), (X[:, 0], y)), 
      shape=(X[:, 0].max()+1, np.unique(y).shape[0])).A
  </code></pre>
  
  <p>But the corresponding table of expected frequencies will most likely have several zero elements.</p>
  
  <p><strong>Edit:</strong></p>
  
  <p>To clarify further, consider the first feature <code>X[:, 0]</code> that is, say, <em>gender</em> and the targets <code>y</code>, say, <em>handedness</em>.</p>
  
  <p>From this we get the cross tabulation</p>
  
  <pre><code>                Right-handed    Left-handed (!right-handed)
  Male            1               2
  Female (!male)  3               4
  </code></pre>
  
  <p>And we can assess the significance of the difference between the two proportions using the Chi-squared test by setting the expected frequency </p>
  
  <p><img src="https://i.stack.imgur.com/sWTpb.png" alt="expfreq"> </p>
  
  <p><code>sklearn.feature_selection.chi2</code> does this directly without resorting to explicitly computing the table and obtains the scores using a more efficient procedure that is equivalent to <code>scipy.stats.chisquare</code>.</p>
  
  <p>After explicitly enumerating the table shown above, I wanted to verify it is consistent with <code>chi2</code> when applying <code>scipy.stats.chi2_contingency</code> and to my dismay, it isn't. I'd like to ask why it isn't.</p>