docstr: Class for training, using and evaluating neural networks described in
  `Distributed Representations of Sentences and Documents <http://arxiv.org/abs/1405.4053v2>`_.
  
  Parameters
  ----------
  documents : iterable of list of :class:`~gensim.models.doc2vec.TaggedDocument`, optional
      Input corpus, can be simply a list of elements, but for larger corpora,consider an iterable that streams
      the documents directly from disk/network. If you don't supply `documents` (or `corpus_file`), the model is
      left uninitialized -- use if you plan to initialize it in some other way.
  corpus_file : str, optional
      Path to a corpus file in :class:`~gensim.models.word2vec.LineSentence` format.
      You may use this argument instead of `documents` to get performance boost. Only one of `documents` or
      `corpus_file` arguments need to be passed (or none of them, in that case, the model is left uninitialized).
      Documents' tags are assigned automatically and are equal to line number, as in
      :class:`~gensim.models.doc2vec.TaggedLineDocument`.
  dm : {1,0}, optional
      Defines the training algorithm. If `dm=1`, 'distributed memory' (PV-DM) is used.
      Otherwise, `distributed bag of words` (PV-DBOW) is employed.
  vector_size : int, optional
      Dimensionality of the feature vectors.
  window : int, optional
      The maximum distance between the current and predicted word within a sentence.
  alpha : float, optional
      The initial learning rate.
  min_alpha : float, optional
      Learning rate will linearly drop to `min_alpha` as training progresses.
  seed : int, optional
      Seed for the random number generator. Initial vectors for each word are seeded with a hash of
      the concatenation of word + `str(seed)`. Note that for a fully deterministically-reproducible run,
      you must also limit the model to a single worker thread (`workers=1`), to eliminate ordering jitter
      from OS thread scheduling.
      In Python 3, reproducibility between interpreter launches also requires use of the `PYTHONHASHSEED`
      environment variable to control hash randomization.
  min_count : int, optional
      Ignores all words with total frequency lower than this.
  max_vocab_size : int, optional
      Limits the RAM during vocabulary building; if there are more unique
      words than this, then prune the infrequent ones. Every 10 million word types need about 1GB of RAM.
      Set to `None` for no limit.
  sample : float, optional
      The threshold for configuring which higher-frequency words are randomly downsampled,
      useful range is (0, 1e-5).
  workers : int, optional
      Use these many worker threads to train the model (=faster training with multicore machines).
  epochs : int, optional
      Number of iterations (epochs) over the corpus. Defaults to 10 for Doc2Vec.
  hs : {1,0}, optional
      If 1, hierarchical softmax will be used for model training.
      If set to 0, and `negative` is non-zero, negative sampling will be used.
  negative : int, optional
      If > 0, negative sampling will be used, the int for negative specifies how many "noise words"
      should be drawn (usually between 5-20).
      If set to 0, no negative sampling is used.
  ns_exponent : float, optional
      The exponent used to shape the negative sampling distribution. A value of 1.0 samples exactly in proportion
      to the frequencies, 0.0 samples all words equally, while a negative value samples low-frequency words more
      than high-frequency words. The popular default value of 0.75 was chosen by the original Word2Vec paper.
      More recently, in https://arxiv.org/abs/1804.04212, Caselles-Dupré, Lesaint, & Royo-Letelier suggest that
      other values may perform better for recommendation applications.
  dm_mean : {1,0}, optional
      If 0, use the sum of the context word vectors. If 1, use the mean.
      Only applies when `dm` is used in non-concatenative mode.
  dm_concat : {1,0}, optional
      If 1, use concatenation of context vectors rather than sum/average;
      Note concatenation results in a much-larger model, as the input
      is no longer the size of one (sampled or arithmetically combined) word vector, but the
      size of the tag(s) and all words in the context strung together.
  dm_tag_count : int, optional
      Expected constant number of document tags per document, when using
      dm_concat mode.
  dbow_words : {1,0}, optional
      If set to 1 trains word-vectors (in skip-gram fashion) simultaneous with DBOW
      doc-vector training; If 0, only trains doc-vectors (faster).
  trim_rule : function, optional
      Vocabulary trimming rule, specifies whether certain words should remain in the vocabulary,
      be trimmed away, or handled using the default (discard if word count < min_count).
      Can be None (min_count will be used, look to :func:`~gensim.utils.keep_vocab_item`),
      or a callable that accepts parameters (word, count, min_count) and returns either
      :attr:`gensim.utils.RULE_DISCARD`, :attr:`gensim.utils.RULE_KEEP` or :attr:`gensim.utils.RULE_DEFAULT`.
      The rule, if given, is only used to prune vocabulary during current method call and is not stored as part
      of the model.
  
      The input parameters are of the following types:
          * `word` (str) - the word we are examining
          * `count` (int) - the word's frequency count in the corpus
          * `min_count` (int) - the minimum count threshold.
  
  callbacks : :obj: `list` of :obj: `~gensim.models.callbacks.CallbackAny2Vec`, optional
      List of callbacks that need to be executed/run at specific stages during training.
  shrink_windows : bool, optional
      New in 4.1. Experimental.
      If True, the effective window size is uniformly sampled from  [1, `window`]
      for each target word during training, to match the original word2vec algorithm's
      approximate weighting of context words by distance. Otherwise, the effective
      window size is always fixed to `window` words to either side.
  
  Some important internal attributes are the following:
  
  Attributes
  ----------
  wv : :class:`~gensim.models.keyedvectors.KeyedVectors`
      This object essentially contains the mapping between words and embeddings. After training, it can be used
      directly to query those embeddings in various ways. See the module level docstring for examples.
  
  dv : :class:`~gensim.models.keyedvectors.KeyedVectors`
      This object contains the paragraph vectors learned from the training data. There will be one such vector
      for each unique document tag supplied during training. They may be individually accessed using the tag
      as an indexed-access key. For example, if one of the training documents used a tag of 'doc003':
  
      .. sourcecode:: pycon
  
          >>> model.dv['doc003']
code: def __init__(
      self,
      documents=None,
      corpus_file=None,
      vector_size=100,
      dm_mean=None,
      dm=1,
      dbow_words=0,
      dm_concat=0,
      dm_tag_count=1,
      dv=None,
      dv_mapfile=None,
      comment=None,
      trim_rule=None,
      callbacks=(),
      window=5,
      epochs=10,
      shrink_windows=True,
      **kwargs,
  ):
      """Class for training, using and evaluating neural networks described in
      `Distributed Representations of Sentences and Documents <http://arxiv.org/abs/1405.4053v2>`_.
  
      Parameters
      ----------
      documents : iterable of list of :class:`~gensim.models.doc2vec.TaggedDocument`, optional
          Input corpus, can be simply a list of elements, but for larger corpora,consider an iterable that streams
          the documents directly from disk/network. If you don't supply `documents` (or `corpus_file`), the model is
          left uninitialized -- use if you plan to initialize it in some other way.
      corpus_file : str, optional
          Path to a corpus file in :class:`~gensim.models.word2vec.LineSentence` format.
          You may use this argument instead of `documents` to get performance boost. Only one of `documents` or
          `corpus_file` arguments need to be passed (or none of them, in that case, the model is left uninitialized).
          Documents' tags are assigned automatically and are equal to line number, as in
          :class:`~gensim.models.doc2vec.TaggedLineDocument`.
      dm : {1,0}, optional
          Defines the training algorithm. If `dm=1`, 'distributed memory' (PV-DM) is used.
          Otherwise, `distributed bag of words` (PV-DBOW) is employed.
      vector_size : int, optional
          Dimensionality of the feature vectors.
      window : int, optional
          The maximum distance between the current and predicted word within a sentence.
      alpha : float, optional
          The initial learning rate.
      min_alpha : float, optional
          Learning rate will linearly drop to `min_alpha` as training progresses.
      seed : int, optional
          Seed for the random number generator. Initial vectors for each word are seeded with a hash of
          the concatenation of word + `str(seed)`. Note that for a fully deterministically-reproducible run,
          you must also limit the model to a single worker thread (`workers=1`), to eliminate ordering jitter
          from OS thread scheduling.
          In Python 3, reproducibility between interpreter launches also requires use of the `PYTHONHASHSEED`
          environment variable to control hash randomization.
      min_count : int, optional
          Ignores all words with total frequency lower than this.
      max_vocab_size : int, optional
          Limits the RAM during vocabulary building; if there are more unique
          words than this, then prune the infrequent ones. Every 10 million word types need about 1GB of RAM.
          Set to `None` for no limit.
      sample : float, optional
          The threshold for configuring which higher-frequency words are randomly downsampled,
          useful range is (0, 1e-5).
      workers : int, optional
          Use these many worker threads to train the model (=faster training with multicore machines).
      epochs : int, optional
          Number of iterations (epochs) over the corpus. Defaults to 10 for Doc2Vec.
      hs : {1,0}, optional
          If 1, hierarchical softmax will be used for model training.
          If set to 0, and `negative` is non-zero, negative sampling will be used.
      negative : int, optional
          If > 0, negative sampling will be used, the int for negative specifies how many "noise words"
          should be drawn (usually between 5-20).
          If set to 0, no negative sampling is used.
      ns_exponent : float, optional
          The exponent used to shape the negative sampling distribution. A value of 1.0 samples exactly in proportion
          to the frequencies, 0.0 samples all words equally, while a negative value samples low-frequency words more
          than high-frequency words. The popular default value of 0.75 was chosen by the original Word2Vec paper.
          More recently, in https://arxiv.org/abs/1804.04212, Caselles-Dupré, Lesaint, & Royo-Letelier suggest that
          other values may perform better for recommendation applications.
      dm_mean : {1,0}, optional
          If 0, use the sum of the context word vectors. If 1, use the mean.
          Only applies when `dm` is used in non-concatenative mode.
      dm_concat : {1,0}, optional
          If 1, use concatenation of context vectors rather than sum/average;
          Note concatenation results in a much-larger model, as the input
          is no longer the size of one (sampled or arithmetically combined) word vector, but the
          size of the tag(s) and all words in the context strung together.
      dm_tag_count : int, optional
          Expected constant number of document tags per document, when using
          dm_concat mode.
      dbow_words : {1,0}, optional
          If set to 1 trains word-vectors (in skip-gram fashion) simultaneous with DBOW
          doc-vector training; If 0, only trains doc-vectors (faster).
      trim_rule : function, optional
          Vocabulary trimming rule, specifies whether certain words should remain in the vocabulary,
          be trimmed away, or handled using the default (discard if word count < min_count).
          Can be None (min_count will be used, look to :func:`~gensim.utils.keep_vocab_item`),
          or a callable that accepts parameters (word, count, min_count) and returns either
          :attr:`gensim.utils.RULE_DISCARD`, :attr:`gensim.utils.RULE_KEEP` or :attr:`gensim.utils.RULE_DEFAULT`.
          The rule, if given, is only used to prune vocabulary during current method call and is not stored as part
          of the model.
  
          The input parameters are of the following types:
              * `word` (str) - the word we are examining
              * `count` (int) - the word's frequency count in the corpus
              * `min_count` (int) - the minimum count threshold.
  
      callbacks : :obj: `list` of :obj: `~gensim.models.callbacks.CallbackAny2Vec`, optional
          List of callbacks that need to be executed/run at specific stages during training.
      shrink_windows : bool, optional
          New in 4.1. Experimental.
          If True, the effective window size is uniformly sampled from  [1, `window`]
          for each target word during training, to match the original word2vec algorithm's
          approximate weighting of context words by distance. Otherwise, the effective
          window size is always fixed to `window` words to either side.
  
      Some important internal attributes are the following:
  
      Attributes
      ----------
      wv : :class:`~gensim.models.keyedvectors.KeyedVectors`
          This object essentially contains the mapping between words and embeddings. After training, it can be used
          directly to query those embeddings in various ways. See the module level docstring for examples.
  
      dv : :class:`~gensim.models.keyedvectors.KeyedVectors`
          This object contains the paragraph vectors learned from the training data. There will be one such vector
          for each unique document tag supplied during training. They may be individually accessed using the tag
          as an indexed-access key. For example, if one of the training documents used a tag of 'doc003':
  
          .. sourcecode:: pycon
  
              >>> model.dv['doc003']
  
      """
      corpus_iterable = documents
  
      if dm_mean is not None:
          self.cbow_mean = dm_mean
  
      self.dbow_words = int(dbow_words)
      self.dm_concat = int(dm_concat)
      self.dm_tag_count = int(dm_tag_count)
      if dm and dm_concat:
          self.layer1_size = (dm_tag_count + (2 * window)) * vector_size
          logger.info(
              "using concatenative %d-dimensional layer1",
              self.layer1_size,
          )
  
      self.vector_size = vector_size
      self.dv = dv or KeyedVectors(
          self.vector_size, mapfile_path=dv_mapfile
      )
      # EXPERIMENTAL lockf feature; create minimal no-op lockf arrays (1 element of 1.0)
      # advanced users should directly resize/adjust as desired after any vocab growth
      self.dv.vectors_lockf = np.ones(
          1, dtype=REAL
      )  # 0.0 values suppress word-backprop-updates; 1.0 allows
  
      super(Doc2Vec, self).__init__(
          sentences=corpus_iterable,
          corpus_file=corpus_file,
          vector_size=self.vector_size,
          sg=(1 + dm) % 2,
          null_word=self.dm_concat,
          callbacks=callbacks,
          window=window,
          epochs=epochs,
          shrink_windows=shrink_windows,
          **kwargs,
      )
  
imports: import logging
  import os
  from collections import namedtuple, defaultdict
  from collections.abc import Iterable
  from timeit import default_timer
  from dataclasses import dataclass
  from numpy import zeros, float32, vstack, integer, dtype
  import numpy
  from gensim import utils, matutils
  from gensim.utils import deprecated
  from gensim.models import Word2Vec, FAST_VERSION
  from gensim.models.keyedvectors import KeyedVectors, pseudorandom_weak_vector
label_desc: Use image node_id 1 to create a new Doc2Vec object from inherited base classes: Word2Vec with example: obj = Doc2Vec(documents, corpus_file, vector_size, dm_mean, dm, dbow_words, dm_concat, dm_tag_count, dv, dv_mapfile, comment, trim_rule, callbacks, window, epochs, shrink_windows)