class_docstr: Handles chunking by adding a new set of offsets system keyed by "chunk"
  Performs chunking based on the following rules
  
  * Select longest axis of the table, rows or columns
  * For this longest axis, chunk such that each chunk's effective num tokens is less than the defined max_length
  * effective num tokens is the number of tokens you get if you duplicate tokens by the number of cells they span.
  * Each chunk get's n_rows_context rows of context from the beginning of the opposite axis to the one being chunked (2 rows of context if columns are chunked)
  * If n_rows_context takes up more than max_length / 2 effective tokens then this context is backed off until the amount of context is less than max_length / 2.
  
  
  There is no invese chunking as TableETL.resolve_preds will simply map directly from chunk to document if chunking is enabled.
code: def _num_tokens(self, rows, effective_tokens=True):
      if effective_tokens:
          return sum(c["num_effective_tokens"] for c in rows)
      return sum(c["num_tokens"] for c in rows)
  
imports: import copy
  import functools
  import os
  import tempfile
  import typing
  import sys
  from finetune.base_models import TableRoBERTa
  from finetune.util.metrics import sequences_overlap
  from finetune.scheduler import Scheduler
  from finetune.encoding.input_encoder import BaseEncoder
  from finetune import SequenceLabeler
label_desc: Use image node_id 5 for calling the TableChunker obj's underlying member method code with example usage: obj._num_tokens(rows, effective_tokens) and returns: sum, sum