learning word embeddings with the embedding layer the simplest way to associate dense vector with word is to choose the vector at random the problem with this approach is that the resulting embedding space has no structurefor instancethe words accurate and exact may end up with completely different embeddingseven though they're interchangeable in most sentences it' difficult for deep neural network to make sense of such noisyunstructured embedding space to get bit more abstractthe geometric relationships between word vectors should reflect the semantic relationships between these words word embeddings are meant to map human language into geometric space for instancein reasonable embedding spaceyou would expect synonyms to be embedded into similar word vectorsand in generalyou would expect the geometric distance (such as distancebetween any two word vectors to relate to the semantic distance between the associated words (words meaning different things are embedded at points far away from each otherwhereas related words are closerin addition to distanceyou may want specific directions in the embedding space to be meaningful to make this clearerlet' look at concrete example in figure four words are embedded on planecatdogwolfand tiger with the vector representations we wolf chose heresome semantic relationships between these tiger words can be encoded as geometric transformations for instancethe same vector allows us to go from cat to tiger dog cat and from dog to wolfthis vector could be interpreted as the "from pet to wild animalvector similarlyanother vector lets us go from dog to cat and from wolf to tigerwhich could be interpreted as "from canine to felinevector figure toy example in real-world word-embedding spacescommon examof word-embedding space ples of meaningful geometric transformations are "gendervectors and "pluralvectors for instanceby adding "femalevector to the vector "king,we obtain the vector "queen by adding "pluralvectorwe obtain "kings word-embedding spaces typically feature thousands of such interpretable and potentially useful vectors is there some ideal word-embedding space that would perfectly map human language and could be used for any natural-language-processing taskpossiblybut we have yet to compute anything of the sort alsothere is no such thing as human language--there are many different languagesand they aren' isomorphicbecause language is the reflection of specific culture and specific context but more pragmaticallywhat makes good word-embedding space depends heavily on your taskthe perfect word-embedding space for an english-language movie-review sentimentanalysis model may look different from the perfect embedding space for an englishlanguage legal-document-classification modelbecause the importance of certain semantic relationships varies from task to task licensed to