def get_pattern_correlations(data):
  """
  Gets the average correlation between all bits in patterns, across the entire
  dataset.  Assumes input is a sparse matrix.  Weighted by pattern rather than
  by bit; this is the average pairwise correlation for every pattern in the
  data, and is not the average pairwise correlation for all bits that ever
  cooccur.  This is a subtle but important difference.
  """

  patterns = [data.rowNonZeros(i)[0] for i in range(data.nRows())]
  dense_data = data.toDense()
  correlations = numpy.corrcoef(dense_data, rowvar = False)
  correlations = numpy.nan_to_num(correlations)
  pattern_correlations = []
  for pattern in patterns:
    pattern_correlations.append([correlations[i, j]
        for i in pattern for j in pattern if i != j])
  return numpy.mean(pattern_correlations)
Gets the average correlation between all bits in patterns, across the entire
  dataset.  Assumes input is a sparse matrix.  Weighted by pattern rather than
  by bit; this is the average pairwise correlation for every pattern in the
  data, and is not the average pairwise correlation for all bits that ever
  cooccur.  This is a subtle but important difference.