class_docstr: This masks out tokens according to the given tokenizer.
  
  The masked variables are
  
  output_type : "string" (default) or "token_ids"
docstr: Compute the clustering of tokens for the given string.
          
code: def clustering(self, s):
      """Compute the clustering of tokens for the given string."""
      self._update_s_cache(s)
      special_tokens = []
      sep_token = getattr_silent(self.tokenizer, "sep_token")
      if sep_token is None:
          special_tokens = []
      else:
          special_tokens = [sep_token]
  
      # convert the text segments to tokens that the partition tree function expects
      tokens = []
      space_end = re.compile(r"^.*\W$")
      letter_start = re.compile(r"^[A-Za-z]")
      for i, v in enumerate(self._segments_s):
          if (
              i > 0
              and space_end.match(self._segments_s[i - 1]) is None
              and letter_start.match(v) is not None
              and tokens[i - 1] != ""
          ):
              tokens.append("##" + v.strip())
          else:
              tokens.append(v.strip())
  
      pt = partition_tree(tokens, special_tokens)
  
      # use the rescaled size of the clusters as their height since the merge scores are just a
      # heuristic and not scaled well
      pt[:, 2] = pt[:, 3]
      pt[:, 2] /= pt[:, 2].max()
  
      return pt
  
imports: import math
  import re
  import numpy
  from .._serializable import Deserializer, Serializer
  from ..utils import safe_isinstance
  from ..utils.transformers import SENTENCEPIECE_TOKENIZERS, getattr_silent, parse_prefix_suffix_for_tokenizer
  from ._masker import Masker
label_desc: Use image node_id 5 for calling the Text obj's underlying member method code with example usage: obj.clustering(s) and returns: pt