Measure of correlation for term frequency I'm trying to write a framework to compare a set of labels such as (for a sample of 5 yes/no answers to a question)  [0, 1, 1, 1, 0] to a series of features to determine correlation.  For numerical non-sparse features, like "number of words" or "average word length", I know I can use a variance-covariance matrix and get a sense for whether or not "number of words" or "average word length" is an informative feature for a model to answer the question.

I'd like to be able to do the same thing for term frequency (let's say using CountVectorizer in scikit-learn), but the resultant covariance matrix will be rather large and will only indicate whether or not that particular term is an informative feature.  How do I get some kind of "collapsed" or "aggregate" measure of correlation?  Is this even possible?
If you did a simple linear regression for each of the 5 outputs, your r-squared would be a good measure of the "aggregate correlation." If you wanted to compare to the other results, it would be helpful to do the same for each of your single features as well.