title: Looking for an effective NLP Phrase Embedding model
body: <p>The goal I want to achieve is to find a good word_and_phrase embedding model that can do:
  (1) For the words and phrases that I am interested in, they have embeddings.
  (2) I can use embeddings to compare similarity between two things(could be word or phrase)</p>
  <p>So far I have tried two paths:</p>
  <p>1: Some Gensim-loaded pre-trained models, for instance:</p>
  <pre><code>from gensim.models.word2vec import Word2Vec
  import gensim.downloader as api
  # download the model and return as object ready for use
  model_glove_twitter = api.load(&quot;fasttext-wiki-news-subwords-300&quot;)
  model_glove_twitter.similarity('computer-science', 'machine-learning')
  </code></pre>
  <p>The problem with this path is that I do not know if a phrase has embedding. For this example, I got this error:</p>
  <pre><code>KeyError: &quot;word 'computer-science' not in vocabulary&quot;
  </code></pre>
  <p>I will have to try different pre-trained models, such as word2vec-google-news-300, glove-wiki-gigaword-300, glove-twitter-200, etc. Results are similar, there are always phrases of interests not having embeddings.</p>
  <ol start="2">
  <li>Then I tried to use some BERT-based sentence embedding method: <a href="https://github.com/UKPLab/sentence-transformers" rel="nofollow noreferrer">https://github.com/UKPLab/sentence-transformers</a>.</li>
  </ol>
  <pre><code>from sentence_transformers import SentenceTransformer
  model = SentenceTransformer('distilbert-base-nli-mean-tokens')
  
  from scipy.spatial.distance import cosine
  
  def cosine_similarity(embedding_1, embedding_2):
      # Calculate the cosine similarity of the two embeddings.
      sim = 1 - cosine(embedding_1, embedding_2)
      print('Cosine similarity: {:.2}'.format(sim))
  
  phrase_1 = 'baby girl'
  phrase_2 = 'annual report'
  embedding_1 = model.encode(phrase_1)
  embedding_2 = model.encode(phrase_2)
  cosine_similarity(embedding_1[0], embedding_2[0])
  </code></pre>
  <p>Using this method I was able to get embeddings for my phrases, but the similarity score was 0.93, which did not seem to be reasonable.</p>
  <p>So what can I try else to achieve the two goals mentioned above?</p>