Problem in constructing co-occurence matrix I have a basic doubt in construction of co-occurence matrix from text. Coocurence matrix X is defined as :

Xij : number of times word i occurs in context of word j in a window of size w.

Now is this context limited to sentence i.e do we context word of a main word in a sentence only or across the whole document?

For ex:

Consider the corpus :

human interface computer
survey user computer system response time
eps user interface system
system human system eps
user response time
trees
graph trees
graph minors trees
graph minors survey
I like graph and stuff
I like trees and stuff
Sometimes I build a graph
Sometimes I build trees


Now for word graph and context window 10 do we look for only 4th last line or do we search for words across line also considering whole doc as one line?
Whether to window only over sentences or over the whole corpus depends on the context of your problem and on the structure of your corpus. In this corpus the order of your sentences does not matter, if they were switched around it would still be the same corpus. This means that you should stick to only the lines and not see it as one line. If your corpus would be a book for example, spilling over sentences makes a lot more sense. If you are unsure, try both, it is just a hyperparameter of your model.