class_docstr: TextSet wraps a set of texts with status.
class_docstr_tok: ["TextSet","wraps","a","set","of","texts","with","status","."]
docstr: Map word tokens to indices.
  Important: Take care that this method behaves a bit differently for training and inference.
  
  ---------------------------------------Training--------------------------------------------
  During the training, you need to generate a new word_index dictionary according to the texts
  you are dealing with. Thus this method will first do the dictionary generation and then
  convert words to indices based on the generated dictionary.
  
  You can specify the following arguments which pose some constraints when generating
  the dictionary.
  In the result dictionary, index will start from 1 and corresponds to the occurrence
  frequency of each word sorted in descending order.
  Here we adopt the convention that index 0 will be reserved for unknown words.
  After word2idx, you can get the generated word_index dictionary by calling 'get_word_index'.
  Also, you can call `save_word_index` to save this word_index dictionary to be used in
  future training.
  
  :param remove_topN: Non-negative int. Remove the topN words with highest frequencies
                      in the case where those are treated as stopwords.
                      Default is 0, namely remove nothing.
  :param max_words_num: Int. The maximum number of words to be taken into consideration.
                        Default is -1, namely all words will be considered.
                        Otherwise, it should be a positive int.
  :param min_freq: Positive int. Only those words with frequency >= min_freq will be taken
                   into consideration.
                   Default is 1, namely all words that occur will be considered.
  :param existing_map: Existing dictionary of word_index if any.
                       Default is None and in this case a new dictionary with index starting
                       from 1 will be generated.
                       If not None, then the generated dictionary will preserve the word_index
                       in existing_map and assign subsequent indices to new words.
  
  ---------------------------------------Inference--------------------------------------------
  During the inference, you are supposed to use exactly the same word_index dictionary as in
  the training stage instead of generating a new one.
  Thus please be aware that you do not need to specify any of the above arguments.
  You need to call `load_word_index` or `set_word_index` beforehand for dictionary loading.
  
  Need to tokenize first.
  See WordIndexer for more details.
  
  :return: TextSet after word2idx.
docstr_tok: ["Map","word","tokens","to","indices",".","Important",":","Take","care","that","this","method","behaves","a","bit","differently","for","training","and","inference",".","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","-Training","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","During","the","training",",","you","need","to","generate","a","new","word_index","dictionary","according","to","the","texts","you","are","dealing","with",".","Thus","this","method","will","first","do","the","dictionary","generation","and","then","convert","words","to","indices","based","on","the","generated","dictionary",".","You","can","specify","the","following","arguments","which","pose","some","constraints","when","generating","the","dictionary",".","In","the","result","dictionary",",","index","will","start","from","1","and","corresponds","to","the","occurrence","frequency","of","each","word","sorted","in","descending","order",".","Here","we","adopt","the","convention","that","index","0","will","be","reserved","for","unknown","words",".","After","word2idx",",","you","can","get","the","generated","word_index","dictionary","by","calling","'get_word_index","'",".","Also",",","you","can","call","`","save_word_index","`","to","save","this","word_index","dictionary","to","be","used","in","future","training",".",":","param","remove_topN",":","Non-negative","int",".","Remove","the","topN","words","with","highest","frequencies","in","the","case","where","those","are","treated","as","stopwords",".","Default","is","0",",","namely","remove","nothing",".",":","param","max_words_num",":","Int",".","The","maximum","number","of","words","to","be","taken","into","consideration",".","Default","is","-1",",","namely","all","words","will","be","considered",".","Otherwise",",","it","should","be","a","positive","int",".",":","param","min_freq",":","Positive","int",".","Only","those","words","with","frequency",">","=","min_freq","will","be","taken","into","consideration",".","Default","is","1",",","namely","all","words","that","occur","will","be","considered",".",":","param","existing_map",":","Existing","dictionary","of","word_index","if","any",".","Default","is","None","and","in","this","case","a","new","dictionary","with","index","starting","from","1","will","be","generated",".","If","not","None",",","then","the","generated","dictionary","will","preserve","the","word_index","in","existing_map","and","assign","subsequent","indices","to","new","words",".","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","-Inference","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","--","During","the","inference",",","you","are","supposed","to","use","exactly","the","same","word_index","dictionary","as","in","the","training","stage","instead","of","generating","a","new","one",".","Thus","please","be","aware","that","you","do","not","need","to","specify","any","of","the","above","arguments",".","You","need","to","call","`","load_word_index","`","or","`","set_word_index","`","beforehand","for","dictionary","loading",".","Need","to","tokenize","first",".","See","WordIndexer","for","more","details",".",":","return",":","TextSet","after","word2idx","."]
code: def word2idx(
      self,
      remove_topN=0,
      max_words_num=-1,
      min_freq=1,
      existing_map=None,
  ):
      """
      Map word tokens to indices.
      Important: Take care that this method behaves a bit differently for training and inference.
  
      ---------------------------------------Training--------------------------------------------
      During the training, you need to generate a new word_index dictionary according to the texts
      you are dealing with. Thus this method will first do the dictionary generation and then
      convert words to indices based on the generated dictionary.
  
      You can specify the following arguments which pose some constraints when generating
      the dictionary.
      In the result dictionary, index will start from 1 and corresponds to the occurrence
      frequency of each word sorted in descending order.
      Here we adopt the convention that index 0 will be reserved for unknown words.
      After word2idx, you can get the generated word_index dictionary by calling 'get_word_index'.
      Also, you can call `save_word_index` to save this word_index dictionary to be used in
      future training.
  
      :param remove_topN: Non-negative int. Remove the topN words with highest frequencies
                          in the case where those are treated as stopwords.
                          Default is 0, namely remove nothing.
      :param max_words_num: Int. The maximum number of words to be taken into consideration.
                            Default is -1, namely all words will be considered.
                            Otherwise, it should be a positive int.
      :param min_freq: Positive int. Only those words with frequency >= min_freq will be taken
                       into consideration.
                       Default is 1, namely all words that occur will be considered.
      :param existing_map: Existing dictionary of word_index if any.
                           Default is None and in this case a new dictionary with index starting
                           from 1 will be generated.
                           If not None, then the generated dictionary will preserve the word_index
                           in existing_map and assign subsequent indices to new words.
  
      ---------------------------------------Inference--------------------------------------------
      During the inference, you are supposed to use exactly the same word_index dictionary as in
      the training stage instead of generating a new one.
      Thus please be aware that you do not need to specify any of the above arguments.
      You need to call `load_word_index` or `set_word_index` beforehand for dictionary loading.
  
      Need to tokenize first.
      See WordIndexer for more details.
  
      :return: TextSet after word2idx.
      """
      jvalue = callZooFunc(
          self.bigdl_type,
          "textSetWord2idx",
          self.value,
          remove_topN,
          max_words_num,
          min_freq,
          existing_map,
      )
      return TextSet(jvalue=jvalue)
  
imports: import six
  from bigdl.util.common import JavaValue
  from zoo.common.utils import callZooFunc
  from pyspark import RDD
label_desc: Use image node_id 19 for calling the TextSet obj's underlying member method code with example usage: obj.word2idx(remove_topN, max_words_num, min_freq, existing_map) and returns: TextSet