title: All words used to used to train Doc2Vec model appear as unknown
body: <p>I'm trying to make an unsupervised text classifier using Doc2Vec. I am getting the error message of &quot;The following keywords from the 'keywords_list' are unknown to the Doc2Vec model and therefore not used to train the model: sport&quot;. The rest of the error message is super long but eventually ends with &quot;cannot compute similarity with no input&quot; which leads me to believe that all of my keywords are not accepted.</p>
  <p>The portion of code where I create the Lbl2Vec model is</p>
  <pre><code> Lbl2Vec_model = Lbl2Vec(keywords_list=list(labels[&quot;keywords&quot;]), tagged_documents=df_pdfs['tagged_docs'], label_names=list(labels[&quot;class_name&quot;]), similarity_threshold=0.43, min_num_docs=10, epochs=10)
  </code></pre>
  <p>I've been following this tutorial <a href="https://towardsdatascience.com/unsupervised-text-classification-with-lbl2vec-6c5e040354de" rel="nofollow noreferrer">https://towardsdatascience.com/unsupervised-text-classification-with-lbl2vec-6c5e040354de</a> but using my own dataset that I load from JSON.</p>
  <p>The entire code is posted below:</p>
  <pre><code>#imports
  from ast import keyword
  import tkinter as tk
  from tkinter import filedialog
  import json
  
  import pandas as pd
  
  import numpy as np
  
  from gensim.utils import simple_preprocess
  from gensim.parsing.preprocessing import strip_tags
  from gensim.models.doc2vec import TaggedDocument
  from lbl2vec import Lbl2Vec
  
  
  class DocumentVectorize:
      def __init__(self):
          pass
      
      #imports data from json file
      def import_from_json (self):
          root = tk.Tk()
          root.withdraw()
          json_file_path = filedialog.askopenfile().name
          with open(json_file_path, &quot;r&quot;) as json_file:
                  try:
                      json_load = json.load(json_file)
                  except:
                      raise ValueError(&quot;No PDFs to convert to JSON&quot;)
          self.pdfs = json_load
  
  
  if __name__ == &quot;__main__&quot;:
  
  #tokenizes documents
      def tokenize(doc):
          return simple_preprocess(strip_tags(doc), deacc=True, min_len=2, max_len=1000000)
  
      #initializes document vectorization class and imports the data from json
      vectorizer = DocumentVectorize()
      vectorizer.import_from_json()
  
      #converts json data to dataframe
      df_pdfs = pd.DataFrame.from_dict(vectorizer.pdfs)
      
      #creates data frame that contains the keywords and their class names for the training
      labels =  {&quot;keywords&quot;: [[&quot;sport&quot;], [&quot;physics&quot;]], &quot;class_name&quot;: [&quot;rec.sport&quot;, &quot;rec.physics&quot;]}
      labels = pd.DataFrame.from_dict(labels)
  
      #applies tokenization to documents
      df_pdfs['tagged_docs'] = df_pdfs.apply(lambda row: TaggedDocument(tokenize(row['text_clean']), [str(row.name)]), axis=1)
      #creates key for documents
      df_pdfs['doc_key'] = df_pdfs.index.astype(str)
      print(df_pdfs.head())
  
      #Initializes Lbl2vec model
      Lbl2Vec_model = Lbl2Vec(keywords_list=list(labels[&quot;keywords&quot;]), tagged_documents=df_pdfs['tagged_docs'], label_names=list(labels[&quot;class_name&quot;]), similarity_threshold=0.43, min_num_docs=10, epochs=10)
  
      #Fits Lbl2vec model to data
      Lbl2Vec_model.fit()
  </code></pre>