class_docstr: Tensorizer for token level classification tasks such as NER, POS etc
  using RoBERTa. Here each token has an associated label and the tensorizer
  should output a label tensor as well. The input for this tensorizer comes
  from the CoNLLUNERDataSource data source.
code: def tensorize(self, batch) -> Tuple[torch.Tensor, ...]:
      tokens, segment_labels, seq_lens, positions, labels = zip(*batch)
      tokens = pad_and_tensorize(tokens, self.vocab.get_pad_index())
      pad_mask = (tokens != self.vocab.get_pad_index()).long()
      segment_labels = pad_and_tensorize(
          segment_labels, self.vocab.get_pad_index()
      )
      positions = pad_and_tensorize(positions)
      padded_labels = pad_and_tensorize(labels, self.labels_pad_idx)
      return tokens, pad_mask, segment_labels, positions, padded_labels
  
imports: import itertools
  from typing import Any, Dict, List, Tuple
  import torch
  from pytext import resources
  from pytext.common.constants import SpecialTokens
  from pytext.config.component import ComponentType, create_component
  from pytext.data.bert_tensorizer import BERTTensorizerBase, BERTTensorizerBaseScriptImpl, build_fairseq_vocab
  from pytext.data.tokenizers import GPT2BPETokenizer, Tokenizer
  from pytext.data.utils import pad_and_tensorize, VocabBuilder, Vocabulary
  from pytext.torchscript.tensorizer import ScriptRoBERTaTensorizer
  from pytext.torchscript.vocab import ScriptVocabulary
  from pytext.utils.file_io import PathManager
label_desc: Use image node_id 6 for calling the RoBERTaTokenLevelTensorizer obj's underlying member method code with example usage: obj.tensorize(batch) and returns: tokens,  pad_mask,  segment_labels,  positions,  padded_labels