class_docstr: Remove words that occur below a certain number of times in a given column.
  This is a common method of cleaning text before it is used, and can increase the
  quality and explainability of the models learned on the transformed data.
  
  RareWordTrimmer can be applied to all the string-, dictionary-, and list-typed
  columns in a given SFrame. Its behavior for each supported input column
  type is as follows. (See :func:`~turicreate.feature_engineering.RareWordTrimmer.transform`
  for usage examples).
  
  * **string** : The string is first tokenized. By default, all letters are
    first converted to lower case, then tokenized by space characters. Each
    token is taken to be a word, and the words occurring below a threshold
    number of times across the entire column are removed, then the remaining
    tokens are concatenated back into a string.
  
  * **list** : Each element of the list must be a string, where each element
    is assumed to be a token. The remaining tokens are then filtered
    by count occurrences and a threshold value.
  
  * **dict** : The method first obtains the list of keys in the dictionary.
    This list is then processed as a standard list, except the value of each
    key must be of integer type and is considered to be the count of that key.
  
  Parameters
  ----------
  features : list[str] | str | None, optional
      Name(s) of feature column(s) to be transformed. If set to None, then all
      feature columns are used.
  
  excluded_features : list[str] | str | None, optional
      Name(s) of feature columns in the input dataset to be ignored. Either
      `excluded_features` or `features` can be passed, but not both.
  
  threshold : int, optional
      The count below which words are removed from the input.
  
  stopwords: list[str], optional
      A manually specified list of stopwords, which are removed regardless
      of count.
  
  to_lower : bool, optional
      Indicates whether to map the input strings to lower case before counting.
  
  delimiters: list[string], optional
      A list of delimiter characters for tokenization. By default, the list
      is defined to be the list of space characters. The user can define
      any custom list of single-character delimiters. Alternatively, setting
      `delimiters=None` will use a Penn treebank type tokenization, which
      is better at handling punctuations. (See reference below for details.)
  
  output_column_prefix : str, optional
      The prefix to use for the column name of each transformed column.
      When provided, the transformation will add columns to the input data,
      where the new name is "`output_column_prefix`.original_column_name".
      If `output_column_prefix=None` (default), then the output column name
      is the same as the original feature column name.
  
  Returns
  -------
  out : RareWordTrimmer
      A RareWordTrimmer feature engineering object which is initialized with
      the defined parameters.
  
  Notes
  -----
  If the SFrame to be transformed already contains a column with the
  designated output column name, then that column will be replaced with the
  new output. In particular, this means that `output_column_prefix=None` will
  overwrite the original feature columns.
  
  References
  ----------
  - `Penn treebank tokenization <https://web.archive.org/web/19970614072242/http://www.cis.upenn.edu:80/~treebank/tokenization.html>`_
  
  See Also
  --------
  turicreate.toolkits.text_analytics.count_words,
  turicreate.toolkits.feature_engineering._ngram_counter.NGramCounter,
  turicreate.toolkits.feature_engineering._tfidf.TFIDF,
  turicreate.toolkits.feature_engineering._tokenizer.Tokenizer,
  turicreate.toolkits.feature_engineering.create
  
  Examples
  --------
  
  .. sourcecode:: python
  
      >>> import turicreate as tc
  
      # Create data.
      >>> sf = tc.SFrame({
      ...    'string': ['sentences Sentences', 'another sentence another year'],
      ...    'dict': [{'bob': 1, 'Bob': 2}, {'a': 0, 'cat': 5}],
      ...    'list': [['one', 'two', 'three', 'Three'], ['a', 'cat', 'Cat']]})
  
      # Create a RareWordTrimmer transformer.
      >>> from turicreate.feature_engineering import RareWordTrimmer
      >>> trimmer = RareWordTrimmer()
  
      # Fit and transform the data.
      >>> transformed_sf = trimmer.fit_transform(sf)
      Columns:
          dict    dict
          list    list
          string  str
  
      Rows: 2
  
      Data:
      +------------+----------------+---------------------+
      |    dict    |      list      |        string       |
      +------------+----------------+---------------------+
      | {'bob': 2} | [three, three] | sentences sentences |
      | {'cat': 5} |   [cat, cat]   |   another another   |
      +------------+----------------+---------------------+
      [2 rows x 3 columns]
  
     # Save the transformer.
     >>> trimmer.save('save-path')
class_docstr_tok: ["Remove","words","that","occur","below","a","certain","number","of","times","in","a","given","column",".","This","is","a","common","method","of","cleaning","text","before","it","is","used",",","and","can","increase","the","quality","and","explainability","of","the","models","learned","on","the","transformed","data",".","RareWordTrimmer","can","be","applied","to","all","the","string-",",","dictionary-",",","and","list-typed","columns","in","a","given","SFrame",".","Its","behavior","for","each","supported","input","column","type","is","as","follows",".","(","See",":","func",":","`","~turicreate.feature_engineering.RareWordTrimmer.transform","`","for","usage","examples",")",".","*","*","*","string","*","*",":","The","string","is","first","tokenized",".","By","default",",","all","letters","are","first","converted","to","lower","case",",","then","tokenized","by","space","characters",".","Each","token","is","taken","to","be","a","word",",","and","the","words","occurring","below","a","threshold","number","of","times","across","the","entire","column","are","removed",",","then","the","remaining","tokens","are","concatenated","back","into","a","string",".","*","*","*","list","*","*",":","Each","element","of","the","list","must","be","a","string",",","where","each","element","is","assumed","to","be","a","token",".","The","remaining","tokens","are","then","filtered","by","count","occurrences","and","a","threshold","value",".","*","*","*","dict","*","*",":","The","method","first","obtains","the","list","of","keys","in","the","dictionary",".","This","list","is","then","processed","as","a","standard","list",",","except","the","value","of","each","key","must","be","of","integer","type","and","is","considered","to","be","the","count","of","that","key",".","Parameters","--","--","--","--","--","features",":","list","[","str","]","|","str","|","None",",","optional","Name","(","s",")","of","feature","column","(","s",")","to","be","transformed",".","If","set","to","None",",","then","all","feature","columns","are","used",".","excluded_features",":","list","[","str","]","|","str","|","None",",","optional","Name","(","s",")","of","feature","columns","in","the","input","dataset","to","be","ignored",".","Either","`","excluded_features","`","or","`","features","`","can","be","passed",",","but","not","both",".","threshold",":","int",",","optional","The","count","below","which","words","are","removed","from","the","input",".","stopwords",":","list","[","str","]",",","optional","A","manually","specified","list","of","stopwords",",","which","are","removed","regardless","of","count",".","to_lower",":","bool",",","optional","Indicates","whether","to","map","the","input","strings","to","lower","case","before","counting",".","delimiters",":","list","[","string","]",",","optional","A","list","of","delimiter","characters","for","tokenization",".","By","default",",","the","list","is","defined","to","be","the","list","of","space","characters",".","The","user","can","define","any","custom","list","of","single-character","delimiters",".","Alternatively",",","setting","`","delimiters=None","`","will","use","a","Penn","treebank","type","tokenization",",","which","is","better","at","handling","punctuations",".","(","See","reference","below","for","details",".",")","output_column_prefix",":","str",",","optional","The","prefix","to","use","for","the","column","name","of","each","transformed","column",".","When","provided",",","the","transformation","will","add","columns","to","the","input","data",",","where","the","new","name","is","``","`","output_column_prefix","`",".original_column_name","''",".","If","`","output_column_prefix=None","`","(","default",")",",","then","the","output","column","name","is","the","same","as","the","original","feature","column","name",".","Returns","--","--","--","-","out",":","RareWordTrimmer","A","RareWordTrimmer","feature","engineering","object","which","is","initialized","with","the","defined","parameters",".","Notes","--","--","-","If","the","SFrame","to","be","transformed","already","contains","a","column","with","the","designated","output","column","name",",","then","that","column","will","be","replaced","with","the","new","output",".","In","particular",",","this","means","that","`","output_column_prefix=None","`","will","overwrite","the","original","feature","columns",".","References","--","--","--","--","--","-","`","Penn","treebank","tokenization","<","https",":","\/\/web.archive.org\/web\/19970614072242\/http",":","\/\/www.cis.upenn.edu:80\/~treebank\/tokenization.html",">","`","_","See","Also","--","--","--","--","turicreate.toolkits.text_analytics.count_words",",","turicreate.toolkits.feature_engineering._ngram_counter.NGramCounter",",","turicreate.toolkits.feature_engineering._tfidf.TFIDF",",","turicreate.toolkits.feature_engineering._tokenizer.Tokenizer",",","turicreate.toolkits.feature_engineering.create","Examples","--","--","--","--","..","sourcecode",":",":","python",">",">",">","import","turicreate","as","tc","#","Create","data",".",">",">",">","sf","=","tc.SFrame","(","{","...","'string","'",":","[","'sentences","Sentences","'",",","'another","sentence","another","year","'","]",",","...","'dict","'",":","[","{","'bob","'",":","1",",","'Bob","'",":","2","}",",","{","'","a","'",":","0",",","'cat","'",":","5","}","]",",","...","'list","'",":","[","[","'one","'",",","'two","'",",","'three","'",",","'Three","'","]",",","[","'","a","'",",","'cat","'",",","'Cat","'","]","]","}",")","#","Create","a","RareWordTrimmer","transformer",".",">",">",">","from","turicreate.feature_engineering","import","RareWordTrimmer",">",">",">","trimmer","=","RareWordTrimmer","(",")","#","Fit","and","transform","the","data",".",">",">",">","transformed_sf","=","trimmer.fit_transform","(","sf",")","Columns",":","dict","dict","list","list","string","str","Rows",":","2","Data",":","+","--","--","--","--","--","--","+","--","--","--","--","--","--","--","--","+","--","--","--","--","--","--","--","--","--","--","-+","|","dict","|","list","|","string","|","+","--","--","--","--","--","--","+","--","--","--","--","--","--","--","--","+","--","--","--","--","--","--","--","--","--","--","-+","|","{","'bob","'",":","2","}","|","[","three",",","three","]","|","sentences","sentences","|","|","{","'cat","'",":","5","}","|","[","cat",",","cat","]","|","another","another","|","+","--","--","--","--","--","--","+","--","--","--","--","--","--","--","--","+","--","--","--","--","--","--","--","--","--","--","-+","[","2","rows","x","3","columns","]","#","Save","the","transformer",".",">",">",">","trimmer.save","(","'save-path","'",")"]
docstr: Returns a structured description of the model, including (where relevant)
  the schema of the training data, description of the training data,
  training statistics, and model hyperparameters.
  
  Returns
  -------
  sections : list (of list of tuples)
      A list of summary sections.
        Each section is a list.
          Each item in a section list is a tuple of the form:
            ('<label>','<field>')
  section_titles: list
      A list of section titles.
        The order matches that of the 'sections' object.
docstr_tok: ["Returns","a","structured","description","of","the","model",",","including","(","where","relevant",")","the","schema","of","the","training","data",",","description","of","the","training","data",",","training","statistics",",","and","model","hyperparameters",".","Returns","--","--","--","-","sections",":","list","(","of","list","of","tuples",")","A","list","of","summary","sections",".","Each","section","is","a","list",".","Each","item","in","a","section","list","is","a","tuple","of","the","form",":","(","'","<","label",">","'",",","'","<","field",">","'",")","section_titles",":","list","A","list","of","section","titles",".","The","order","matches","that","of","the","'sections","'","object","."]
code: def _get_summary_struct(self):
      """
      Returns a structured description of the model, including (where relevant)
      the schema of the training data, description of the training data,
      training statistics, and model hyperparameters.
  
      Returns
      -------
      sections : list (of list of tuples)
          A list of summary sections.
            Each section is a list.
              Each item in a section list is a tuple of the form:
                ('<label>','<field>')
      section_titles: list
          A list of section titles.
            The order matches that of the 'sections' object.
      """
      _features = _precomputed_field(
          _internal_utils.pretty_print_list(self.get("features"))
      )
      _exclude = _precomputed_field(
          _internal_utils.pretty_print_list(
              self.get("excluded_features")
          )
      )
      _stopwords = _precomputed_field(
          _internal_utils.pretty_print_list(self.get("stopwords"))
      )
  
      fields = [
          ("Features", _features),
          ("Excluded features", _exclude),
          ("Output column name", "output_column_prefix"),
          ("Word count threshold", "threshold"),
          ("Manually specified stopwords", _stopwords),
          ("Whether to convert to lowercase", "to_lower"),
          ("Delimiters", "delimiters"),
      ]
      section_titles = ["Model fields"]
  
      return ([fields], section_titles)
  
file_path: turicreate/src/python/turicreate/toolkits/_feature_engineering/_word_trimmer.py
imports: from __future__ import print_function
  from __future__ import division
  from __future__ import absolute_import
  import turicreate
  from ._feature_engineering import Transformer
  from turicreate.toolkits._internal_utils import _toolkit_repr_print
  from turicreate.toolkits._internal_utils import _precomputed_field
  from turicreate.toolkits._private_utils import _summarize_accessible_fields
  from turicreate.util import _raise_error_if_not_of_type
  from .None import _internal_utils
label_desc: Use image node_id 2 for calling the RareWordTrimmer obj's underlying member method code with example usage: obj._get_summary_struct() and returns: list,  section_titles