class_docstr: Word Embeddings lookup annotator that maps tokens to vectors.
  
  For instantiated/pretrained models, see :class:`.WordEmbeddingsModel`.
  
  A custom token lookup dictionary for embeddings can be set with
  :meth:`.setStoragePath`. Each line of the provided file needs to have a
  token, followed by their vector representation, delimited by a spaces::
  
      ...
      are 0.39658191506190343 0.630968081620067 0.5393722253731201 0.8428180123359783
      were 0.7535235923631415 0.9699218875629833 0.10397182122983872 0.11833962569383116
      stress 0.0492683418305907 0.9415954572751959 0.47624463167525755 0.16790967216778263
      induced 0.1535748762292387 0.33498936903209897 0.9235178224122094 0.1158772920395934
      ...
  
  
  If a token is not found in the dictionary, then the result will be a zero
  vector of the same dimension. Statistics about the rate of converted tokens,
  can be retrieved with :meth:`WordEmbeddingsModel.withCoverageColumn()
  <sparknlp.annotator.WordEmbeddingsModel.withCoverageColumn>` and
  :meth:`WordEmbeddingsModel.overallCoverage()
  <sparknlp.annotator.WordEmbeddingsModel.overallCoverage>`.
  
  For extended examples of usage, see the `Examples
  <https://github.com/JohnSnowLabs/spark-nlp/blob/master/scala/training/NerDL/win/customNerDlPipeline/CustomForNerDLPipeline.java>`__.
  
  ====================== ======================
  Input Annotation types Output Annotation type
  ====================== ======================
  ``DOCUMENT, TOKEN``    ``WORD_EMBEDDINGS``
  ====================== ======================
  
  Parameters
  ----------
  writeBufferSize
      Buffer size limit before dumping to disk storage while writing, by
      default 10000
  readCacheSize
      Cache size for items retrieved from storage. Increase for performance
      but higher memory consumption
  
  Examples
  --------
  In this example, the file ``random_embeddings_dim4.txt`` has the form of the
  content above.
  
  >>> import sparknlp
  >>> from sparknlp.base import *
  >>> from sparknlp.annotator import *
  >>> from pyspark.ml import Pipeline
  >>> documentAssembler = DocumentAssembler() \
  ...     .setInputCol("text") \
  ...     .setOutputCol("document")
  >>> tokenizer = Tokenizer() \
  ...     .setInputCols(["document"]) \
  ...     .setOutputCol("token")
  >>> embeddings = WordEmbeddings() \
  ...     .setStoragePath("src/test/resources/random_embeddings_dim4.txt", ReadAs.TEXT) \
  ...     .setStorageRef("glove_4d") \
  ...     .setDimension(4) \
  ...     .setInputCols(["document", "token"]) \
  ...     .setOutputCol("embeddings")
  >>> embeddingsFinisher = EmbeddingsFinisher() \
  ...     .setInputCols(["embeddings"]) \
  ...     .setOutputCols("finished_embeddings") \
  ...     .setOutputAsVector(True) \
  ...     .setCleanAnnotations(False)
  >>> pipeline = Pipeline() \
  ...     .setStages([
  ...       documentAssembler,
  ...       tokenizer,
  ...       embeddings,
  ...       embeddingsFinisher
  ...     ])
  >>> data = spark.createDataFrame([["The patient was diagnosed with diabetes."]]).toDF("text")
  >>> result = pipeline.fit(data).transform(data)
  >>> result.selectExpr("explode(finished_embeddings) as result").show(truncate=False)
  +----------------------------------------------------------------------------------+
  |result                                                                            |
  +----------------------------------------------------------------------------------+
  |[0.9439099431037903,0.4707513153553009,0.806300163269043,0.16176554560661316]     |
  |[0.7966810464859009,0.5551124811172485,0.8861005902290344,0.28284206986427307]    |
  |[0.025029370561242104,0.35177749395370483,0.052506182342767715,0.1887107789516449]|
  |[0.08617766946554184,0.8399239182472229,0.5395117998123169,0.7864698767662048]    |
  |[0.6599600911140442,0.16109347343444824,0.6041093468666077,0.8913561105728149]    |
  |[0.5955275893211365,0.01899011991918087,0.4397728443145752,0.8911281824111938]    |
  |[0.9840458631515503,0.7599489092826843,0.9417727589607239,0.8624503016471863]     |
  +----------------------------------------------------------------------------------+
  
  See Also
  --------
  SentenceEmbeddings : to combine embeddings into a sentence-level representation
class_docstr_tok: ["Word","Embeddings","lookup","annotator","that","maps","tokens","to","vectors",".","For","instantiated\/pretrained","models",",","see",":","class",":","`",".WordEmbeddingsModel","`",".","A","custom","token","lookup","dictionary","for","embeddings","can","be","set","with",":","meth",":","`",".setStoragePath","`",".","Each","line","of","the","provided","file","needs","to","have","a","token",",","followed","by","their","vector","representation",",","delimited","by","a","spaces",":",":","...","are","0.39658191506190343","0.630968081620067","0.5393722253731201","0.8428180123359783","were","0.7535235923631415","0.9699218875629833","0.10397182122983872","0.11833962569383116","stress","0.0492683418305907","0.9415954572751959","0.47624463167525755","0.16790967216778263","induced","0.1535748762292387","0.33498936903209897","0.9235178224122094","0.1158772920395934","...","If","a","token","is","not","found","in","the","dictionary",",","then","the","result","will","be","a","zero","vector","of","the","same","dimension",".","Statistics","about","the","rate","of","converted","tokens",",","can","be","retrieved","with",":","meth",":","`","WordEmbeddingsModel.withCoverageColumn","(",")","<","sparknlp.annotator.WordEmbeddingsModel.withCoverageColumn",">","`","and",":","meth",":","`","WordEmbeddingsModel.overallCoverage","(",")","<","sparknlp.annotator.WordEmbeddingsModel.overallCoverage",">","`",".","For","extended","examples","of","usage",",","see","the","`","Examples","<","https",":","\/\/github.com\/JohnSnowLabs\/spark-nlp\/blob\/master\/scala\/training\/NerDL\/win\/customNerDlPipeline\/CustomForNerDLPipeline.java",">","`","__",".","======================","======================","Input","Annotation","types","Output","Annotation","type","======================","======================","``","DOCUMENT",",","TOKEN","``","``","WORD_EMBEDDINGS","``","======================","======================","Parameters","--","--","--","--","--","writeBufferSize","Buffer","size","limit","before","dumping","to","disk","storage","while","writing",",","by","default","10000","readCacheSize","Cache","size","for","items","retrieved","from","storage",".","Increase","for","performance","but","higher","memory","consumption","Examples","--","--","--","--","In","this","example",",","the","file","``","random_embeddings_dim4.txt","``","has","the","form","of","the","content","above",".",">",">",">","import","sparknlp",">",">",">","from","sparknlp.base","import","*",">",">",">","from","sparknlp.annotator","import","*",">",">",">","from","pyspark.ml","import","Pipeline",">",">",">","documentAssembler","=","DocumentAssembler","(",")","\\","...",".setInputCol","(","``","text","''",")","\\","...",".setOutputCol","(","``","document","''",")",">",">",">","tokenizer","=","Tokenizer","(",")","\\","...",".setInputCols","(","[","``","document","''","]",")","\\","...",".setOutputCol","(","``","token","''",")",">",">",">","embeddings","=","WordEmbeddings","(",")","\\","...",".setStoragePath","(","``","src\/test\/resources\/random_embeddings_dim4.txt","''",",","ReadAs.TEXT",")","\\","...",".setStorageRef","(","``","glove_4d","''",")","\\","...",".setDimension","(","4",")","\\","...",".setInputCols","(","[","``","document","''",",","``","token","''","]",")","\\","...",".setOutputCol","(","``","embeddings","''",")",">",">",">","embeddingsFinisher","=","EmbeddingsFinisher","(",")","\\","...",".setInputCols","(","[","``","embeddings","''","]",")","\\","...",".setOutputCols","(","``","finished_embeddings","''",")","\\","...",".setOutputAsVector","(","True",")","\\","...",".setCleanAnnotations","(","False",")",">",">",">","pipeline","=","Pipeline","(",")","\\","...",".setStages","(","[","...","documentAssembler",",","...","tokenizer",",","...","embeddings",",","...","embeddingsFinisher","...","]",")",">",">",">","data","=","spark.createDataFrame","(","[","[","``","The","patient","was","diagnosed","with","diabetes",".","``","]","]",")",".toDF","(","``","text","''",")",">",">",">","result","=","pipeline.fit","(","data",")",".transform","(","data",")",">",">",">","result.selectExpr","(","``","explode","(","finished_embeddings",")","as","result","''",")",".show","(","truncate=False",")","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","+","|result","|","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","+","|","[","0.9439099431037903,0.4707513153553009,0.806300163269043,0.16176554560661316","]","|","|","[","0.7966810464859009,0.5551124811172485,0.8861005902290344,0.28284206986427307","]","|","|","[","0.025029370561242104,0.35177749395370483,0.052506182342767715,0.1887107789516449","]","|","|","[","0.08617766946554184,0.8399239182472229,0.5395117998123169,0.7864698767662048","]","|","|","[","0.6599600911140442,0.16109347343444824,0.6041093468666077,0.8913561105728149","]","|","|","[","0.5955275893211365,0.01899011991918087,0.4397728443145752,0.8911281824111938","]","|","|","[","0.9840458631515503,0.7599489092826843,0.9417727589607239,0.8624503016471863","]","|","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","+","See","Also","--","--","--","--","SentenceEmbeddings",":","to","combine","embeddings","into","a","sentence-level","representation"]
code: def __init__(self):
      super(WordEmbeddings, self).__init__(
          classname="com.johnsnowlabs.nlp.embeddings.WordEmbeddings"
      )
      self._setDefault(
          caseSensitive=False,
          writeBufferSize=10000,
          storageRef=self.uid,
      )
  
file_path: spark-nlp/python/sparknlp/annotator/embeddings/word_embeddings.py
imports: from sparknlp.common import *
label_desc: Use image node_id 3 to create a new WordEmbeddings object from inherited base classes: AnnotatorApproach, HasEmbeddingsProperties, HasStorage with example: obj = WordEmbeddings()