class_docstr: A DataProvider that splits a string tensor into individual tokens and
  returns the tokens and the length.
  Optionally prepends or appends special tokens.
  
  Args:
    delimiter: Delimiter to split on. Must be a single character.
    tokens_feature_name: A descriptive feature name for the token values
    length_feature_name: A descriptive feature name for the length value
class_docstr_tok: ["A","DataProvider","that","splits","a","string","tensor","into","individual","tokens","and","returns","the","tokens","and","the","length",".","Optionally","prepends","or","appends","special","tokens",".","Args",":","delimiter",":","Delimiter","to","split","on",".","Must","be","a","single","character",".","tokens_feature_name",":","A","descriptive","feature","name","for","the","token","values","length_feature_name",":","A","descriptive","feature","name","for","the","length","value"]
code: def __init__(
      self,
      delimiter=" ",
      tokens_feature_name="tokens",
      length_feature_name="length",
      prepend_token=None,
      append_token=None,
  ):
      self.delimiter = delimiter
      self.tokens_feature_name = tokens_feature_name
      self.length_feature_name = length_feature_name
      self.prepend_token = prepend_token
      self.append_token = append_token
  
file_path: seq2seq/seq2seq/data/split_tokens_decoder.py
imports: from __future__ import absolute_import
  from __future__ import division
  from __future__ import print_function
  from __future__ import unicode_literals
  import tensorflow
  from tensorflow.contrib.slim.python.slim.data import data_decoder
label_desc: Use image node_id 1 to create a new SplitTokensDecoder object from inherited base classes: data_decoder with example: obj = SplitTokensDecoder(delimiter, tokens_feature_name, length_feature_name, prepend_token, append_token)