A more recent and popular approach for representing sentences is Sentence-BERT, or SentenceTransformers, which modifies pre-trained BERT with the use of siamese and triplet network structures. Software Software for training and using word embeddings includes Tomas Mikolov's Word2vec, Stanford University's GloVe, GN-GloVe, Flair embeddings, AllenNLP's ELMo, BERT, fastText, Gensim, Indra, and Deeplearning4j. Principal Component Analysis (PCA) and T-Distributed Stochastic Neighbour Embedding (t-SNE) are both used to reduce the dimensionality of word vector spaces and visualize word embeddings and clusters. Examples of application For instance, the fastText is also used to calculate word embeddings for text corpora in Sketch Engine that are available online. Possible Ethical Implications Word embeddings may contain the biases and stereotypes contained in the trained dataset, as Bolukbasi et al. points out in the 2016 paper “Man is to Computer Programmer as Women is to Homemaker? Debiasing Word Embeddings”, a publicly available and popular word2vec embedding trained on a commonly used data corpus of Google News texts, which is written by professional journalists, still shows disproportionate word associations that reflects gender and racial biases when extracting word analogies (Bolukbasi et al. 2016). For example, one of the analogies generated using the aforementioned word embedding is “man is to computer programmer as woman is to homemaker”. The application of these trained word embeddings without careful oversight could possibly amplify existing bias in society, which is reflected in the training data. Further, word embeddings could even amplify these biases (Zhao et al. 2017). Given word embeddings popular use in NLP applications such as search ranking, CV parsing and recommendation systems, the biases that exist in pre-trained word embeddings may have profound impact that remains invisible or hard to measure. See also Brown clustering Distributional–relational database References Language modeling Artificial neural networks Natural language processing Computational linguistics Semantic relations