title: How to construct PPMI matrix from a text corpus?
body: <p>I am trying to use an SVD model for word embedding on the Brown corpus. For this, I want to first generate a word-word co-occurence matrix and then convert to PPMI matrix for the SVD matrix multiplication process.</p>
  
  <p>I have tried to create a co-occurence using SkLearn CountVectorizer </p>
  
  <pre><code>count_model = CountVectorizer(ngram_range=(1,1))
  
  X = count_model.fit_transform(corpus)
  X[X &gt; 0] = 1
  Xc = (X.T * X)
  Xc.setdiag(0)
  print(Xc.todense())
  </code></pre>
  
  <p>But:</p>
  
  <p>(1) Am not sure how I can control the context window with this method? I want to experiment with various context sizes and see how the impact the process.</p>
  
  <p>(2) How do I then compute the PPMI properly assuming that 
  PMI(a, b) = log p(a, b)/p(a)p(b)</p>
  
  <p>Any help on the thought process and implementation would be greatly appreciated!</p>
  
  <p>Thanks (-:</p>