sentence1: we initialize our model with 300-dimensional word2vec toolkit vectors generated by a continuous skip-gram model trained on around 100 billion words from the google news corpus .
sentence2: we present the text to the encoder as a sequence of word2vec word embeddings from a word2vec model trained on the hrwac corpus .