deep learning for text and sequences using pretrained word embeddings sometimesyou have so little training data available that you can' use your data alone to learn an appropriate task-specific embedding of your vocabulary what do you do theninstead of learning word embeddings jointly with the problem you want to solveyou can load embedding vectors from precomputed embedding space that you know is highly structured and exhibits useful properties--that captures generic aspects of language structure the rationale behind using pretrained word embeddings in natural-language processing is much the same as for using pretrained convnets in image classificationyou don' have enough data available to learn truly powerful features on your ownbut you expect the features that you need to be fairly generic--that iscommon visual features or semantic features in this caseit makes sense to reuse features learned on different problem such word embeddings are generally computed using word-occurrence statistics (observations about what words co-occur in sentences or documents)using variety of techniquessome involving neural networksothers not the idea of denselowdimensional embedding space for wordscomputed in an unsupervised waywas initially explored by bengio et al in the early , but it only started to take off in research and industry applications after the release of one of the most famous and successful word-embedding schemesthe word vec algorithm (archive/ /word vec)developed by tomas mikolov at google in word vec dimensions capture specific semantic propertiessuch as gender there are various precomputed databases of word embeddings that you can download and use in keras embedding layer word vec is one of them another popular one is called global vectors for word representation (gloveedu/projects/glove)which was developed by stanford researchers in this embedding technique is based on factorizing matrix of word co-occurrence statistics its developers have made available precomputed embeddings for millions of english tokensobtained from wikipedia data and common crawl data let' look at how you can get started using glove embeddings in keras model the same method is valid for word vec embeddings or any other word-embedding database you'll also use this example to refresh the text-tokenization techniques introduced few paragraphs agoyou'll start from raw text and work your way up putting it all togetherfrom raw text to word embeddings you'll use model similar to the one we just went overembedding sentences in sequences of vectorsflattening themand training dense layer on top but you'll do so using pretrained word embeddingsand instead of using the pretokenized imdb data packaged in kerasyou'll start from scratch by downloading the original text data yoshua bengio et al neural probabilistic language models (springer licensed to