title: How to properly sample from a numpy.random.multivariate_normal (positive-semidefinite covariance matrix issue)
body: <p>I'm hoping to generate new "fake" data from the data I already have with <code>numpy.random.multivariate_normal</code>.</p>
  
  <p>With n samples and d features in an n x d pandas DataFrame: </p>
  
  <pre><code>means = data.mean(axis=0)
  covariances = data.cov()
  variances = data.var()
  means.shape, covariances.shape, variances.shape
  
  &gt;&gt;&gt; ((16349,), (16349, 16349), (16349,))
  </code></pre>
  
  <p>This looks fine, but the covariance matrix <code>covariances</code> isn't positive semidefinite, which is a requirement of <code>numpy.random.multivariate_normal</code>. </p>
  
  <pre><code>x = np.linalg.eigvals(covariances)
  
  np.all(x &gt;= 0)
  &gt;&gt;&gt; False
  
  len([y for y in x if y &lt; 0])  # negative eigenvalues
  &gt;&gt;&gt; 4396
  
  len([y for y in x if y &gt; 0])  # positive eigenvalues
  &gt;&gt;&gt; 4585
  
  len([y for y in x if y == 0])  # zero eigenvalues.
  &gt;&gt;&gt; 7368
  </code></pre>
  
  <p>However, <a href="https://en.wikipedia.org/wiki/Covariance_matrix" rel="nofollow noreferrer">Wikipedia says</a> </p>
  
  <blockquote>
    <p>In addition, every covariance matrix is positive semi-definite.</p>
  </blockquote>
  
  <p>Which leads me to wonder whether <a href="http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.cov.html" rel="nofollow noreferrer">pandas.DataFrame.cov</a> gets you a real covariance matrix. <a href="https://github.com/pandas-dev/pandas/blob/v0.19.2/pandas/core/frame.py#L4757-L4791" rel="nofollow noreferrer">Here's the function's implementation.</a> It seems to mostly defer to <a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.cov.html" rel="nofollow noreferrer">numpy.cov</a> which also seems to promise a covariance matrix. </p>
  
  <p>Can someone clear this up for me? Why is <code>pandas.DataFrame.covs()</code> not positive semidefinite?</p>
  
  <h3>Updated question:</h3>
  
  <p>From the first answer, it seems like all the negative eigenvalues are tiny. The author of that answer suggests clipping these eigenvalues, but it's still unclear to me how to sensibly generate a proper covariance matrix with this information. </p>
  
  <p>I can imagine using <code>pd.DataFrame.cov()</code>, doing eigendecomposition to get eigenvectors and values, clipping the values, and then multiplying those matrices to get a new covariance matrix, but that seems quite precarious. Is that done in practice, or is there a better way? </p>