class_docstr: Holds a batch of input and output objects for a transformer model. The
  data can then be split to a list of `TransformerData` objects to associate
  the outputs to each `Doc` in the batch.
  
  Attributes
  ----------
  spans (List[List[Span]]): The batch of input spans. The outer list refers
      to the Doc objects in the batch, and the inner list are the spans for
      that `Doc`. Note that spans are allowed to overlap or exclude tokens,
      but each Span can only refer to one Doc (by definition). This means that
      within a Doc, the regions of the output tensors that correspond to each
      Span may overlap or have gaps, but for each Doc, there is a non-overlapping
      contiguous slice of the outputs.
  wordpieces (WordpieceBatch): Token data from the Huggingface tokenizer.
  model_output (ModelOutput): The output of the transformer model.
  align (Ragged): Alignment from the spaCy tokenization to the wordpieces.
      This is a ragged array, where align.lengths[i] indicates the number of
      wordpiece tokens that token i aligns against. The actual indices are
      provided at align[i].dataXd.
docstr: Deprecated. Dict formatted version of the self.wordpieces data,
  with values converted to PyTorch tensors.
docstr_tok: ["Deprecated",".","Dict","formatted","version","of","the","self.wordpieces","data",",","with","values","converted","to","PyTorch","tensors","."]
code: def tokens(self) -> Dict[str, Any]:
      """Deprecated. Dict formatted version of the self.wordpieces data,
      with values converted to PyTorch tensors.
      """
      return self.wordpieces.to_hf_dict()
  
imports: from typing import Optional, List, Dict, Any, Union, Tuple, cast
  from dataclasses import dataclass, field
  import torch
  import numpy
  from transformers.tokenization_utils import BatchEncoding
  from transformers.file_utils import ModelOutput
  from transformers.modeling_outputs import BaseModelOutput
  from thinc.types import Ragged, Floats2d, Floats3d, FloatsXd, Ints1d, Ints2d
  from thinc.api import NumpyOps, get_array_module, xp2torch, torch2xp
  from spacy.tokens import Span
  import srsly
  from .util import transpose_list
  from .align import get_token_positions
label_desc: Use image node_id 3 for calling the FullTransformerBatch obj's underlying member method code with example usage: obj.tokens() and returns: self