title: Is it possible to export and use a spaCy NER model without an vocab and inject tokens/vectors on the fly?
body: <p><strong>INTRO</strong></p>
  <p>My question originates a bit far from the title but in essence sums up well what I'm stuck with at the moment.</p>
  <p>I need to integrate a NER model of spacy as part of a complex and distributed NLP pipeline and what I'm doing to do so is:</p>
  <ol>
  <li><p>Train a new NER model based on <code>en_core_web_lg</code> model to recognize also my custom entities in NER task</p>
  </li>
  <li><p>Save the model skipping the vocabulary to save disk space and memory usage</p>
  </li>
  <li><p>Finally load the model to run some inference, using tokens and vectors that somebody pre-computed before in my pipeline, instead of compute it again using the model vocabulary (standard way).</p>
  </li>
  </ol>
  <p>The reason why I am saving the model without the vocab is because in my distributed pipeline one of the first things that is done is tokenize / vectorize the text so that the rest of the tasks have this input.</p>
  <p>→ Before continuing I want to clarify that in the standard way (saving the vocab), I could train my custom NER, save / load and run an inference without major problems with a very good accuracy.</p>
  <p>After that and reading the <a href="https://v2.spacy.io/api/language#to_disk" rel="nofollow noreferrer">spaCy documentation</a>, I found that it is possible to save my model without the vocabulary, and you can even <a href="https://v2.spacy.io/api/doc#init" rel="nofollow noreferrer">build the Doc</a> from a list of tokens and a custom vocabulary (in my case an empty vocab). Also, I was also able to set the document vectors using those that someone previously calculated for me in my pipeline.</p>
  <p>However, when I save the model inside the <code>ner/cfg</code> file I see that there is a reference to the vectors on which the NER model was trained (en_core_web_lg.vectors):</p>
  <pre><code>{
     &quot;disable&quot;:[
       &quot;tagger&quot;,
       &quot;parser&quot;
     ],
     &quot;beam_width&quot;:1,
     &quot;beam_density&quot;:0.0,
     &quot;beam_update_prob&quot;:1.0,
     &quot;cnn_maxout_pieces&quot;:3,
     &quot;nr_feature_tokens&quot;:6,
     &quot;deprecation_fixes&quot;:{
        &quot;vectors_name&quot;:&quot;en_core_web_lg.vectors&quot;
     },
     &quot;nr_class&quot;:86,
     &quot;hidden_depth&quot;:1,
     &quot;token_vector_width&quot;:96,
     &quot;hidden_width&quot;:64,
     &quot;maxout_pieces&quot;:2,
     &quot;pretrained_vectors&quot;:&quot;en_core_web_lg.vectors&quot;,
     &quot;bilstm_depth&quot;:0,
     &quot;self_attn_depth&quot;:0,
     &quot;conv_depth&quot;:4,
     &quot;conv_window&quot;:1,
     &quot;embed_size&quot;:2000
   }
  </code></pre>
  <p>This reference is the cause of an error when I try to load the model without having those vectors in memory (in other words without having that vocab loaded).</p>
  <p>If I delete those references in the cfg file, the model loads correctly and I can run inferences using my vectors but the predictions obtained are very different from the ones I obtained with my first model (with the original vocab) and contain several errors.</p>
  <p><strong>QUESTION</strong></p>
  <p>This brings me to my original question: is it possible to save the NER model with an empty vocab and then run an inference with a SpaCy Doc built somehow from the tokens and vectors previously calculated in my pipeline?</p>
  <p>Thanks a lot in advance!</p>
  <p>BTW I'm using <code>spaCy 2.3.7</code> and I put below some snippets of my code to clarify:</p>
  <p><strong>1. Training</strong>:</p>
  <pre><code>    nlp = spacy.load(&quot;en_core_web_lg&quot;, disable=['tagger', 'parser'])
  
      ner = nlp.get_pipe('ner')
      ner.add_label(&quot;FOO_ENTITY&quot;)
      ner.add_label(&quot;BAR_ENTITY&quot;)
      ner.add_label(&quot;COOL_ENTITY&quot;)
  
      # Start the training
      optimizer = nlp.begin_training()
  
      # Loop for EPOCHS iterations
      losses_hist = []
      for itn in range(30):
          # Shuffle the training data
          random.shuffle(Xy_train)
          losses = {}
  
          # Batch the examples and iterate over them
          for batch in spacy.util.minibatch(Xy_train, size=32):
              texts = [text for text, entities, _ in batch]
              golds = [{'entities': entities} for text, entities, _ in batch]
  
              # Update the model
              nlp.update(docs=texts, golds=golds, losses=losses)
  
          print(losses)
          losses_hist.append(losses)
  </code></pre>
  <p><strong>2.a Run inference (standard)</strong>:</p>
  <pre><code>    # I already have the text split in tokens
      doc = Doc(nlp.vocab, words=tokens)  # Create doc from tokens
      ner = nlp.get_pipe(&quot;ner&quot;)
      doc = ner(doc)  # Call NER step for doc
  
      for ent in doc.ents:
          print(f&quot;value: {ent.text}, start: {ent.start_char}, end: {ent.end_char}, entity: {ent.label_}&quot;)
  </code></pre>
  <p><strong>2.b Run inference with external vectors</strong></p>
  <pre><code>    # I already have the text split in tokens and their vectors
      vectors = Vectors(data=embeddings, keys=tokens)
      nlp.vocab.vectors = vectors
      doc = Doc(nlp.vocab, words=tokens)
      
      ner = nlp.get_pipe(&quot;ner&quot;)
      doc = ner(doc)  # Call NER step for doc
  
      for ent in doc.ents:
          print(f&quot;value: {ent.text}, start: {ent.start_char}, end: {ent.end_char}, entity: {ent.label_}&quot;)
  </code></pre>
  <p><strong>3. Save / Load model</strong>:</p>
  <pre><code># Save model
  nlp.to_disk(str(dir_)
  
  # Load model
  nlp = spacy.load(str(dir_), exclude=['vocab']) 
  </code></pre>