class_docstr: An abstract base class for splitting text.
  
  A Splitter is a module that splits strings into pieces.  Generally, the pieces
  returned by a splitter correspond to substrings of the original string, and
  can be encoded using either strings or integer ids (where integer ids could be
  created by hashing strings or by looking them up in a fixed vocabulary table
  that maps strings to ids).
  
  Each Splitter subclass must implement a `split` method, which subdivides
  each string in an input Tensor into pieces.  E.g.:
  
  >>> class SimpleSplitter(tf_text.Splitter):
  ...   def split(self, input):
  ...     return tf.strings.split(input)
  >>> print(SimpleSplitter().split(["hello world", "this is a test"]))
  <tf.RaggedTensor [[b'hello', b'world'], [b'this', b'is', b'a', b'test']]>
class_docstr_tok: ["An","abstract","base","class","for","splitting","text",".","A","Splitter","is","a","module","that","splits","strings","into","pieces",".","Generally",",","the","pieces","returned","by","a","splitter","correspond","to","substrings","of","the","original","string",",","and","can","be","encoded","using","either","strings","or","integer","ids","(","where","integer","ids","could","be","created","by","hashing","strings","or","by","looking","them","up","in","a","fixed","vocabulary","table","that","maps","strings","to","ids",")",".","Each","Splitter","subclass","must","implement","a","`","split","`","method",",","which","subdivides","each","string","in","an","input","Tensor","into","pieces",".","E.g",".",":",">",">",">","class","SimpleSplitter","(","tf_text.Splitter",")",":","...","def","split","(","self",",","input",")",":","...","return","tf.strings.split","(","input",")",">",">",">","print","(","SimpleSplitter","(",")",".split","(","[","``","hello","world","''",",","``","this","is","a","test","''","]",")",")","<","tf.RaggedTensor","[","[","b'hello","'",",","b'world","'","]",",","[","b'this","'",",","b'is","'",",","b","'","a","'",",","b'test","'","]","]",">"]
docstr: Splits the input tensor into pieces.
  
  Generally, the pieces returned by a splitter correspond to substrings of the
  original string, and can be encoded using either strings or integer ids.
  
  Example:
  
  >>> print(tf_text.WhitespaceTokenizer().split("small medium large"))
  tf.Tensor([b'small' b'medium' b'large'], shape=(3,), dtype=string)
  
  Args:
    input: An N-dimensional UTF-8 string (or optionally integer) `Tensor` or
      `RaggedTensor`.
  
  Returns:
    An N+1-dimensional UTF-8 string or integer `Tensor` or `RaggedTensor`.
    For each string from the input tensor, the final, extra dimension contains
    the pieces that string was split into.
docstr_tok: ["Splits","the","input","tensor","into","pieces",".","Generally",",","the","pieces","returned","by","a","splitter","correspond","to","substrings","of","the","original","string",",","and","can","be","encoded","using","either","strings","or","integer","ids",".","Example",":",">",">",">","print","(","tf_text.WhitespaceTokenizer","(",")",".split","(","``","small","medium","large","''",")",")","tf.Tensor","(","[","b'small","'","b'medium","'","b'large","'","]",",","shape=","(","3",",",")",",","dtype=string",")","Args",":","input",":","An","N-dimensional","UTF-8","string","(","or","optionally","integer",")","`","Tensor","`","or","`","RaggedTensor","`",".","Returns",":","An","N+1-dimensional","UTF-8","string","or","integer","`","Tensor","`","or","`","RaggedTensor","`",".","For","each","string","from","the","input","tensor",",","the","final",",","extra","dimension","contains","the","pieces","that","string","was","split","into","."]
code: def split(self, input):  # pylint: disable=redefined-builtin
      """Splits the input tensor into pieces.
  
      Generally, the pieces returned by a splitter correspond to substrings of the
      original string, and can be encoded using either strings or integer ids.
  
      Example:
  
      >>> print(tf_text.WhitespaceTokenizer().split("small medium large"))
      tf.Tensor([b'small' b'medium' b'large'], shape=(3,), dtype=string)
  
      Args:
        input: An N-dimensional UTF-8 string (or optionally integer) `Tensor` or
          `RaggedTensor`.
  
      Returns:
        An N+1-dimensional UTF-8 string or integer `Tensor` or `RaggedTensor`.
        For each string from the input tensor, the final, extra dimension contains
        the pieces that string was split into.
      """
      raise NotImplementedError("Abstract method")
  
file_path: text/tensorflow_text/python/ops/splitter.py
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  import abc
  from tensorflow.python.module import module
label_desc: Use image node_id 1 for calling the Splitter obj's underlying member method code with example usage: obj.split(input) without return types