However, we found that explicitly initializing these embeddings with vectors pre-trained over a large collection of unlabeled data significantly improved performance (see Section "Effects of initialized embeddings and corrupt-sampling schemes" ). 
 Question: What is the new initialization method proposed in this paper?
They initialize their word and entity embeddings with vectors pre-trained over a large corpus of unlabeled data.