Title: Toward Incorporation of Relevant Documents in word2vec,
Abstract: Recent advances in neural word embedding provide significant benefit to
various information retrieval tasks. However as shown by recent studies,
adapting the embedding models for the needs of IR tasks can bring considerable
further improvements. The embedding models in general define the term
relatedness by exploiting the terms' co-occurrences in short-window contexts.
An alternative (and well-studied) approach in IR for related terms to a query
is using local information i.e. a set of top-retrieved documents. In view of
these two methods of term relatedness, in this work, we report our study on
incorporating the local information of the query in the word embeddings. One
main challenge in this direction is that the dense vectors of word embeddings
and their estimation of term-to-term relatedness remain difficult to interpret
and hard to analyze. As an alternative, explicit word representations propose
vectors whose dimensions are easily interpretable, and recent methods show
competitive performance to the dense vectors. We introduce a neural-based
explicit representation, rooted in the conceptual ideas of the word2vec
Skip-Gram model. The method provides interpretable explicit vectors while
keeping the effectiveness of the Skip-Gram model. The evaluation of various
explicit representations on word association collections shows that the newly
proposed method out- performs the state-of-the-art explicit representations
when tasked with ranking highly similar terms. Based on the introduced ex-
plicit representation, we discuss our approaches on integrating local documents
in globally-trained embedding models and discuss the preliminary results.