docstr: This function contains logic for converting tokens into ids based on
  the specified vocab. It also outputs, for each instance, the vectors
  needed to run the actual model.
  
  Args:
      per_sentence_tokens: list of tokens per sentence level in one row,
      each token represented by token string, start and end indices.
  
  Returns:
      tokens: List[int], a list of token ids, concatenate all
      sentences token ids.
      segment_labels: List[int], denotes each token belong to
      which sentence.
      seq_len: int, tokens length
      positions: List[int], token positions
code: def numberize(
      self, per_sentence_tokens: List[List[Tuple[str, int, int]]]
  ) -> Tuple[List[int], List[int], int, List[int]]:
      """
      This function contains logic for converting tokens into ids based on
      the specified vocab. It also outputs, for each instance, the vectors
      needed to run the actual model.
  
      Args:
          per_sentence_tokens: list of tokens per sentence level in one row,
          each token represented by token string, start and end indices.
  
      Returns:
          tokens: List[int], a list of token ids, concatenate all
          sentences token ids.
          segment_labels: List[int], denotes each token belong to
          which sentence.
          seq_len: int, tokens length
          positions: List[int], token positions
      """
      tokens: List[int] = []
      segment_labels: List[int] = []
      seq_len: int = 0
      positions: List[int] = []
  
      for idx, single_sentence_tokens in enumerate(per_sentence_tokens):
          lookup_ids: List[int] = self._lookup_tokens(
              single_sentence_tokens
          )[0]
          lookup_ids = self._wrap_numberized_tokens(lookup_ids, idx)
  
          tokens.extend(lookup_ids)
          segment_labels.extend([idx] * len(lookup_ids))
  
      seq_len = len(tokens)
      positions = [i for i in range(seq_len)]
      return tokens, segment_labels, seq_len, positions
  
imports: from typing import Any, Dict, List, Optional, Tuple
  import torch
  from fairseq.data.dictionary import Dictionary
  from fairseq.data.legacy.masked_lm_dictionary import BertDictionary
  from pytext import resources
  from pytext.common.constants import Token
  from pytext.config.component import ComponentType, create_component
  from pytext.data.tensorizers import Tensorizer, TensorizerScriptImpl
  from pytext.data.tokenizers import Tokenizer, WordPieceTokenizer
  from pytext.data.utils import BOS, EOS, MASK, PAD, UNK, Vocabulary
  from pytext.torchscript.tensorizer.tensorizer import VocabLookup
  from pytext.torchscript.utils import pad_2d_mask, ScriptBatchInput
  from pytext.torchscript.vocab import ScriptVocabulary
  from pytext.utils.file_io import PathManager
  from pytext.utils.lazy import lazy_property
  from pytorch.text.fb.utils.transform_utils import pad_2d
label_desc: Use image node_id 4 for calling the BERTTensorizerBaseScriptImpl obj's underlying member method code with example usage: obj.numberize(per_sentence_tokens) and returns: tokens,  segment_labels,  seq_len,  positions