class_docstr: Special purpose Data object which assumes a single text tensorizer.  Packs
  tokens into a square batch with no padding.  Used for LM training. The object
  also takes in an optional language argument which is used for cross-lingual
  LM training.
class_docstr_tok: ["Special","purpose","Data","object","which","assumes","a","single","text","tensorizer",".","Packs","tokens","into","a","square","batch","with","no","padding",".","Used","for","LM","training",".","The","object","also","takes","in","an","optional","language","argument","which","is","used","for","cross-lingual","LM","training","."]
docstr: The tensorize function for different tensorizers takes in different
  number of inputs which may be arranged differently. This function formats
  the output dict to conform to the expectations of the tensorizer.
  In case of the XLMTensorizer, we also need to create a new positions list
  which goes from 0 to seq_len.
docstr_tok: ["The","tensorize","function","for","different","tensorizers","takes","in","different","number","of","inputs","which","may","be","arranged","differently",".","This","function","formats","the","output","dict","to","conform","to","the","expectations","of","the","tensorizer",".","In","case","of","the","XLMTensorizer",",","we","also","need","to","create","a","new","positions","list","which","goes","from","0","to","seq_len","."]
code: def _format_output_row(self, tokens, segment_labels, seq_len):
      """
      The tensorize function for different tensorizers takes in different
      number of inputs which may be arranged differently. This function formats
      the output dict to conform to the expectations of the tensorizer.
      In case of the XLMTensorizer, we also need to create a new positions list
      which goes from 0 to seq_len.
      """
      if isinstance(self.tensorizer, BERTTensorizerBase):
          positions = [index for index in range(seq_len)]
          return {
              self.tensorizer_name: (
                  tokens,
                  segment_labels,
                  seq_len,
                  positions,
              )
          }
      elif isinstance(self.tensorizer, TokenTensorizer):
          # dummy token_ranges
          return {
              self.tensorizer_name: (
                  tokens,
                  seq_len,
                  [(-1, -1)] * seq_len,
              )
          }
      else:
          raise NotImplementedError(
              "PackedLMData only supports BERTTensorizer and TokenTensorizer."
          )
  
file_path: pytext/pytext/data/packed_lm_data.py
imports: from typing import Dict, List, Optional, Type
  from pytext.common.constants import Stage
  from pytext.data import Batcher, Data
  from pytext.data.bert_tensorizer import BERTTensorizerBase
  from pytext.data.data import RowData
  from pytext.data.sources import DataSource
  from pytext.data.tensorizers import Tensorizer, TokenTensorizer
label_desc: Use image node_id 4 for calling the PackedLMData obj's underlying member method code with example usage: obj._format_output_row(tokens, segment_labels, seq_len) and returns: dict, dict