title: is seaborn confidence interval computed correctly?
body: <p>First, I must admit that my statistics knowledge is rusty at best: even when it was shining new, it's not a discipline I particularly liked, which means I had a hard time making sense of it.</p>
  
  <p>Nevertheless, I took a look at how the <a href="http://seaborn.pydata.org/generated/seaborn.barplot.html#seaborn.barplot" rel="noreferrer">barplot</a> graphs were calculating error bars, and was surprised to find a "confidence interval" (CI) used instead of (the more common) standard deviation. Researching more CI led me to this <a href="https://en.wikipedia.org/wiki/Confidence_interval" rel="noreferrer">wikipedia article</a> which seems to say that, basically, a CI is computed as:</p>
  
  <p><img src="https://user-images.githubusercontent.com/796623/30232889-5d0e8242-94c0-11e7-9d39-89907c04a5a8.png" alt="mean minus 1.96 times stdev over sqrt(n)"></p>
  
  <p><img src="https://user-images.githubusercontent.com/796623/30232893-6331ca80-94c0-11e7-9972-ecd8d2324a03.png" alt="mean plus 1.96 times stdev over sqrt(n)"></p>
  
  <p>Or, in pseudocode:</p>
  
  <pre><code>def ci_wp(a):
      """calculate confidence interval using Wikipedia's formula"""
      m = np.mean(a)
      s = 1.96*np.std(a)/np.sqrt(len(a))
      return m - s, m + s
  </code></pre>
  
  <p>But what we find in <a href="https://github.com/mwaskom/seaborn/blob/master/seaborn/utils.py#L326" rel="noreferrer">seaborn/utils.py</a> is:</p>
  
  <pre><code>def ci(a, which=95, axis=None):
      """Return a percentile range from an array of values."""
      p = 50 - which / 2, 50 + which / 2
      return percentiles(a, p, axis)
  </code></pre>
  
  <p>Now maybe I'm missing this completely, but this seems just like a completely different calculation than the one proposed by Wikipedia. Can anyone explain this discrepancy?</p>
  
  <p>To give another example, from comments, why do we get so different results between:</p>
  
  <pre><code> &gt;&gt;&gt; sb.utils.ci(np.arange(100))
   array([ 2.475, 96.525])
  
   &gt;&gt;&gt; ci_wp(np.arange(100))
   [43.842250270646467,55.157749729353533]
  </code></pre>
  
  <p>And to compare with other statistical tools:</p>
  
  <pre><code> def ci_std(a):
       """calculate margin of error using standard deviation"""
       m = np.mean(a)
       s = np.std(a)
       return m-s, m+s
  
   def ci_sem(a):
       """calculate margin of error using standard error of the mean"""
       m = np.mean(a)
       s = sp.stats.sem(a)
       return m-s, m+s
  </code></pre>
  
  <p>Which gives us:</p>
  
  <pre><code>&gt;&gt;&gt; ci_sem(np.arange(100))
  (46.598850802411796, 52.401149197588204)
  
  &gt;&gt;&gt; ci_std(np.arange(100))
  (20.633929952277882, 78.366070047722118)
  </code></pre>
  
  <p>Or with a random sample:</p>
  
  <pre><code>rng = np.random.RandomState(10)
  a = rng.normal(size=100)
  print sb.utils.ci(a)
  print ci_wp(a)
  print ci_sem(a)
  print ci_std(a)
  </code></pre>
  
  <p>... which yields:</p>
  
  <pre><code>[-1.9667006   2.19502303]
  (-0.1101230745774124, 0.26895640045116026)
  (-0.017774461397903049, 0.17660778727165088)
  (-0.88762281417683186, 1.0464561400505796)
  </code></pre>
  
  <p>Why are Seaborn's numbers so radically different from the other results?</p>