Title: Beyond Word Embeddings: Learning Entity and Concept Representations from Large Scale Knowledge Bases,
Abstract: Text representations using neural word embeddings have proven effective in
many NLP applications. Recent researches adapt the traditional word embedding
models to learn vectors of multiword expressions (concepts/entities). However,
these methods are limited to textual knowledge bases (e.g., Wikipedia). In this
paper, we propose a novel and simple technique for integrating the knowledge
about concepts from two large scale knowledge bases of different structure
(Wikipedia and Probase) in order to learn concept representations. We adapt the
efficient skip-gram model to seamlessly learn from the knowledge in Wikipedia
text and Probase concept graph. We evaluate our concept embedding models on two
tasks: (1) analogical reasoning, where we achieve a state-of-the-art
performance of 91% on semantic analogies, (2) concept categorization, where we
achieve a state-of-the-art performance on two benchmark datasets achieving
categorization accuracy of 100% on one and 98% on the other. Additionally, we
present a case study to evaluate our model on unsupervised argument type
identification for neural semantic parsing. We demonstrate the competitive
accuracy of our unsupervised method and its ability to better generalize to out
of vocabulary entity mentions compared to the tedious and error prone methods
which depend on gazetteers and regular expressions.