class_docstr: Customtokenizer which will tokenize the input text as words or phases and convert the words (str) to an index (int) using the vocab.
  If you would like tokens, please use `hub.BertTokenizer`.
docstr: Converts a token string (or a sequence of tokens) in a single integer id
  (or a sequence of ids), using the vocabulary.
docstr_tok: ["Converts","a","token","string","(","or","a","sequence","of","tokens",")","in","a","single","integer","id","(","or","a","sequence","of","ids",")",",","using","the","vocabulary","."]
code: def convert_tokens_to_ids(self, tokens: List[str]):
      """Converts a token string (or a sequence of tokens) in a single integer id
      (or a sequence of ids), using the vocabulary.
      """
      if tokens is None:
          return None
  
      if isinstance(tokens, str):
          return self._convert_token_to_id(tokens)
  
      ids = []
      for token in tokens:
          wid = self._convert_token_to_id(token)
          if wid is not None:
              ids.append(wid)
      return ids
  
imports: import collections
  import inspect
  import os
  from typing import Callable, List, Union
  import paddlehub
  from paddlehub.utils.log import logger
  from paddlehub.text.bert_tokenizer import BasicTokenizer
  from paddlehub.text.utils import load_vocab, whitespace_tokenize
label_desc: Use image node_id 8 for calling the CustomTokenizer obj's underlying member method code with example usage: obj.convert_tokens_to_ids(tokens) and returns: ids, None, self