Why is T test reweighting on a word X word co-occurrence matrix so effective?
IT is very similiar to PMI, here you just expand it to the whole dictionary matrix (matrix representation of the whole vocabulary), normalize it by subtracting quantitive representation of the sum of words found in row i column j and than standardize. (Like when using sklearn Standardize(), similiar atleast)
Intuition? Well why is tf-idf working (generally as text quantification technique), you are focusing on essential n-grams and minimising away the rest, with this re-weighting you are getting close to tf-idf representation in a sence.