deep learning for text and sequences word_index tokenizer word_index print('found % unique tokens len(word_index)data pad_sequences(sequencesmaxlen=maxlenlabels np asarray(labelsprint('shape of data tensor:'data shapeprint('shape of label tensor:'labels shapeindices np arange(data shape[ ]np random shuffle(indicesdata data[indiceslabels labels[indicessplits the data into training set and validation setbut first shuffles the databecause you're starting with data in which samples are ordered (all negative firstthen all positivex_train data[:training_samplesy_train labels[:training_samplesx_val data[training_samplestraining_samples validation_samplesy_val labels[training_samplestraining_samples validation_samplesdownloading the glove word embeddings go to embeddings from english wikipedia it' an mb zip file called glove zipcontaining -dimensional embedding vectors for , words (or nonword tokensunzip it preprocessing the embeddings let' parse the unzipped file ( txt fileto build an index that maps words (as stringsto their vector representation (as number vectorslisting parsing he glove word-embeddings file glove_dir '/users/fchollet/downloads/glove bembeddings_index { open(os path join(glove_dir'glove txt')for line in fvalues line split(word values[ coefs np asarray(values[ :]dtype='float 'embeddings_index[wordcoefs close(print('found % word vectors len(embeddings_index)nextyou'll build an embedding matrix that you can load into an embedding layer it must be matrix of shape (max_wordsembedding_dim)where each entry contains the embedding_dim-dimensional vector for the word of index in the reference word index (built during tokenizationnote that index isn' supposed to stand for any word or token--it' placeholder licensed to