class_docstr: Produces inputs and answer spans for Squad.
class_docstr_tok: ["Produces","inputs","and","answer","spans","for","Squad","."]
code: def tensorize(self, batch):
      (
          doc_tokens,
          doc_seq_len,
          ques_tokens,
          ques_seq_len,
          answer_start_idx,
          answer_end_idx,
      ) = zip(*batch)
      doc_tokens = pad_and_tensorize(
          doc_tokens, self.vocab.get_pad_index()
      )
      doc_mask = (
          doc_tokens == self.vocab.get_pad_index()
      ).byte()  # 1 => pad
      ques_tokens = pad_and_tensorize(
          ques_tokens, self.vocab.get_pad_index()
      )
      ques_mask = (
          ques_tokens == self.vocab.get_pad_index()
      ).byte()  # 1 => pad
      answer_start_idx = pad_and_tensorize(
          answer_start_idx, self.SPAN_PAD_IDX
      )
      answer_end_idx = pad_and_tensorize(
          answer_end_idx, self.SPAN_PAD_IDX
      )
  
      # doc_tokens must be returned as the first element for
      # SquadMetricReporter._add_decoded_answer_batch_stats() to work
      return (
          doc_tokens,
          pad_and_tensorize(doc_seq_len),
          doc_mask,
          ques_tokens,
          pad_and_tensorize(ques_seq_len),
          ques_mask,
          answer_start_idx,
          answer_end_idx,
      )
  
imports: from typing import List
  import torch
  from pytext.common.constants import SpecialTokens
  from pytext.config.component import ComponentType, create_component
  from pytext.data.tensorizers import TokenTensorizer
  from pytext.data.tokenizers import Tokenizer, WordPieceTokenizer
  from pytext.data.utils import pad_and_tensorize, VocabBuilder, Vocabulary
label_desc: Use image node_id 6 for calling the SquadTensorizer obj's underlying member method code with example usage: obj.tensorize(batch) and returns: doc_tokens,  pad_and_tensorize,  doc_mask,  ques_tokens,  pad_and_tensorize,  ques_mask,  answer_start_idx,  answer_end_idx