class_docstr: Annotator that cleans out tokens. Requires stems, hence tokens. Removes
  all dirty characters from text following a regex pattern and transforms
  words based on a provided dictionary
  
  For extended examples of usage, see the `Examples
  <https://github.com/JohnSnowLabs/spark-nlp/blob/master/examples/python/annotation/text/english/document-normalizer/document_normalizer_notebook.ipynb>`__.
  
  ====================== ======================
  Input Annotation types Output Annotation type
  ====================== ======================
  ``TOKEN``              ``TOKEN``
  ====================== ======================
  
  Parameters
  ----------
  cleanupPatterns
      Normalization regex patterns which match will be removed from token, by default ['[^\pL+]']
  lowercase
      Whether to convert strings to lowercase, by default False
  slangDictionary
      Slang dictionary is a delimited text. needs 'delimiter' in options
  minLength
      The minimum allowed length for each token, by default 0
  maxLength
      The maximum allowed length for each token
  
  Examples
  --------
  >>> import sparknlp
  >>> from sparknlp.base import *
  >>> from sparknlp.annotator import *
  >>> from pyspark.ml import Pipeline
  >>> documentAssembler = DocumentAssembler() \
  ...     .setInputCol("text") \
  ...     .setOutputCol("document")
  >>> tokenizer = Tokenizer() \
  ...     .setInputCols(["document"]) \
  ...     .setOutputCol("token")
  >>> normalizer = Normalizer() \
  ...     .setInputCols(["token"]) \
  ...     .setOutputCol("normalized") \
  ...     .setLowercase(True) \
  ...     .setCleanupPatterns(["""[^\w\d\s]"""])
  
  The pattern removes punctuations (keeps alphanumeric chars). If we don't set
  CleanupPatterns, it will only keep alphabet letters ([^A-Za-z])
  
  >>> pipeline = Pipeline().setStages([
  ...     documentAssembler,
  ...     tokenizer,
  ...     normalizer
  ... ])
  >>> data = spark.createDataFrame([["John and Peter are brothers. However they don't support each other that much."]]) \
  ...     .toDF("text")
  >>> result = pipeline.fit(data).transform(data)
  >>> result.selectExpr("normalized.result").show(truncate = False)
  +----------------------------------------------------------------------------------------+
  |result                                                                                  |
  +----------------------------------------------------------------------------------------+
  |[john, and, peter, are, brothers, however, they, dont, support, each, other, that, much]|
  +----------------------------------------------------------------------------------------+
class_docstr_tok: ["Annotator","that","cleans","out","tokens",".","Requires","stems",",","hence","tokens",".","Removes","all","dirty","characters","from","text","following","a","regex","pattern","and","transforms","words","based","on","a","provided","dictionary","For","extended","examples","of","usage",",","see","the","`","Examples","<","https",":","\/\/github.com\/JohnSnowLabs\/spark-nlp\/blob\/master\/examples\/python\/annotation\/text\/english\/document-normalizer\/document_normalizer_notebook.ipynb",">","`","__",".","======================","======================","Input","Annotation","types","Output","Annotation","type","======================","======================","``","TOKEN","``","``","TOKEN","``","======================","======================","Parameters","--","--","--","--","--","cleanupPatterns","Normalization","regex","patterns","which","match","will","be","removed","from","token",",","by","default","[","'","[","^\\pL+","]","'","]","lowercase","Whether","to","convert","strings","to","lowercase",",","by","default","False","slangDictionary","Slang","dictionary","is","a","delimited","text",".","needs","'delimiter","'","in","options","minLength","The","minimum","allowed","length","for","each","token",",","by","default","0","maxLength","The","maximum","allowed","length","for","each","token","Examples","--","--","--","--",">",">",">","import","sparknlp",">",">",">","from","sparknlp.base","import","*",">",">",">","from","sparknlp.annotator","import","*",">",">",">","from","pyspark.ml","import","Pipeline",">",">",">","documentAssembler","=","DocumentAssembler","(",")","\\","...",".setInputCol","(","``","text","''",")","\\","...",".setOutputCol","(","``","document","''",")",">",">",">","tokenizer","=","Tokenizer","(",")","\\","...",".setInputCols","(","[","``","document","''","]",")","\\","...",".setOutputCol","(","``","token","''",")",">",">",">","normalizer","=","Normalizer","(",")","\\","...",".setInputCols","(","[","``","token","''","]",")","\\","...",".setOutputCol","(","``","normalized","''",")","\\","...",".setLowercase","(","True",")","\\","...",".setCleanupPatterns","(","[","``","''","''","[","^\\w\\d\\s","]","''","''","''","]",")","The","pattern","removes","punctuations","(","keeps","alphanumeric","chars",")",".","If","we","do","n't","set","CleanupPatterns",",","it","will","only","keep","alphabet","letters","(","[","^A-Za-z","]",")",">",">",">","pipeline","=","Pipeline","(",")",".setStages","(","[","...","documentAssembler",",","...","tokenizer",",","...","normalizer","...","]",")",">",">",">","data","=","spark.createDataFrame","(","[","[","``","John","and","Peter","are","brothers",".","However","they","do","n't","support","each","other","that","much",".","''","]","]",")","\\","...",".toDF","(","``","text","''",")",">",">",">","result","=","pipeline.fit","(","data",")",".transform","(","data",")",">",">",">","result.selectExpr","(","``","normalized.result","''",")",".show","(","truncate","=","False",")","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","+","|result","|","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","+","|","[","john",",","and",",","peter",",","are",",","brothers",",","however",",","they",",","dont",",","support",",","each",",","other",",","that",",","much","]","|","+","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","+"]
docstr: Sets the maximum allowed length for each token.
  
  Parameters
  ----------
  value : int
      Maximum allowed length for each token
docstr_tok: ["Sets","the","maximum","allowed","length","for","each","token",".","Parameters","--","--","--","--","--","value",":","int","Maximum","allowed","length","for","each","token"]
code: def setMaxLength(self, value):
      """Sets the maximum allowed length for each token.
  
      Parameters
      ----------
      value : int
          Maximum allowed length for each token
      """
      return self._set(maxLength=value)
  
file_path: spark-nlp/python/sparknlp/annotator/normalizer.py
imports: from sparknlp.common import *
label_desc: Use image node_id 6 for calling the Normalizer obj's underlying member method code with example usage: obj.setMaxLength(value) and returns: self