class_docstr: Abstract Layer for tensorflow_text tokenizers.
  
  Input shape:
    N-D (2D as default) tensor with shape: `(batch_size, input_length)`.
  
  Output shape:
    (N+1)-D (3D as default) tensor with shape:
      `(batch_size, input_length, token_dim)`.
code: def call(self, text_to_be_tokenized):
      if self._squeeze_token_dim and (
          text_to_be_tokenized.shape.ndims > 1
      ):
          text_to_be_tokenized = tf.compat.v1.squeeze(
              text_to_be_tokenized, axis=1
          )
      text = self._tokenizer.tokenize(text_to_be_tokenized)
      if self._pad_value is not None:
          text = ragged_conversion_ops.to_tensor(
              text, default_value=self._pad_value
          )
      return text
  
imports: import os
  import tensorflow
  from tensorflow.python.ops import lookup_ops
  from tensorflow.python.ops.ragged import ragged_conversion_ops
  from tensorflow_text.python.ops import unicode_script_tokenizer
  from tensorflow_text.python.ops import whitespace_tokenizer
  from tensorflow_text.python.ops import wordpiece_tokenizer
label_desc: Use image node_id 4 for calling the TokenizerBase obj's underlying member method code with example usage: obj.call(text_to_be_tokenized) and returns: text