BertPunc (punctuation restoration with BERT)
I am also confused by the method of BertPunc. Actually I can't believe it really works. Beacause the method use BertLanguageModel to do the job, and LanguseModel's main capabiltiy is to predict the next word of a word. So also use your example,  BertPunc inputs 8 words, the output of bert should be 8 next words of the inputs. But BertPunc added a Linear layer to only get 1 output value, which is to predict punctuation.

But I think we should use bert the way like the graph below: