class_docstr: Special purpose Data object which assumes a single text tensorizer.  Packs
  tokens into a square batch with no padding.  Used for LM training. The object
  also takes in an optional language argument which is used for cross-lingual
  LM training.
class_docstr_tok: ["Special","purpose","Data","object","which","assumes","a","single","text","tensorizer",".","Packs","tokens","into","a","square","batch","with","no","padding",".","Used","for","LM","training",".","The","object","also","takes","in","an","optional","language","argument","which","is","used","for","cross-lingual","LM","training","."]
code: def numberize_rows(self, rows):
      last_row = None
      """
          This function does the actual packing. It processes rows until we obtain
          a block of data with length = max_seq_len.
          """
      for row in rows:
          last_row = row
  
          # if the packedLM object has a language member then a cross-lingual
          # LM is being trained using monolingual data.
          # Add this language to the row since the underlying
          # tensorizer needs this to generate language embeddings (used as
          # segment_labels below)
          if self.language:
              row["language"] = self.language
  
          tokens, segment_labels, seq_len = self._parse_row(row)
          remaining = (
              self.max_seq_len - len(self.remainder["tokens"]) - 1
          )
          while remaining < len(tokens):
              self.remainder["tokens"].extend(tokens[:remaining])
              self.remainder["segment_labels"].extend(
                  segment_labels[:remaining]
              )
              tokens = tokens[remaining:]
              segment_labels = segment_labels[remaining:]
              # packed LM data doesn't respect data cardinality,
              # therefore, it stores the row at the start position,
              # instead of the exact corresponding row.
              yield self._yield_and_reset(row)
              remaining = self.max_seq_len - 1
          self.remainder["tokens"].extend(tokens)
          self.remainder["segment_labels"].extend(segment_labels)
      if len(self.remainder["tokens"]):
          yield self._yield_and_reset(last_row)
  
file_path: pytext/pytext/data/packed_lm_data.py
imports: from typing import Dict, List, Optional, Type
  from pytext.common.constants import Stage
  from pytext.data import Batcher, Data
  from pytext.data.bert_tensorizer import BERTTensorizerBase
  from pytext.data.data import RowData
  from pytext.data.sources import DataSource
  from pytext.data.tensorizers import Tensorizer, TokenTensorizer
label_desc: Use image node_id 6 for calling the PackedLMData obj's underlying member method code with example usage: obj.numberize_rows(rows) without return types