sentence1: thus , we pre-train the embeddings on a huge unlabeled data , the chinese wikipedia corpus , with word2vec toolkit .
sentence2: for word-level embeddings , we pre-train the word vectors using word2vec on the gigaword corpus mentioned in section 4 , and the text of the training dataset .