class_docstr: Handles chunking by adding a new set of offsets system keyed by "chunk"
  Performs chunking based on the following rules
  
  * Select longest axis of the table, rows or columns
  * For this longest axis, chunk such that each chunk's effective num tokens is less than the defined max_length
  * effective num tokens is the number of tokens you get if you duplicate tokens by the number of cells they span.
  * Each chunk get's n_rows_context rows of context from the beginning of the opposite axis to the one being chunked (2 rows of context if columns are chunked)
  * If n_rows_context takes up more than max_length / 2 effective tokens then this context is backed off until the amount of context is less than max_length / 2.
  
  
  There is no invese chunking as TableETL.resolve_preds will simply map directly from chunk to document if chunking is enabled.
code: def combine_row_spans(self, row_spans, token_spans):
      def mark_token(t):
          t["used"] = True
          return t
  
      total_num_tokens = 0
      combined_rows = []
      for row in row_spans:
          row_out = []
          for span in sorted(row, key=lambda x: x["start"]):
              if row_out and row_out[-1]["end"] + 1 == span["start"]:
                  row_out[-1]["end"] = span["end"]
              else:
                  row_out.append(span)
          for row_span in row_out:
              row_span["num_tokens"] = len(
                  [
                      mark_token(t)
                      for t in token_spans
                      if overlaps_token(row_span, t)
                  ]
              )
              # Accounts for the fact that cells are duplicated when they span cells.
              row_span["num_effective_tokens"] = (
                  row_span["num_tokens"] * row_span["max_cell_span"]
              )
              total_num_tokens += row_span["num_tokens"]
          combined_rows.append(
              {
                  "num_tokens": sum(r["num_tokens"] for r in row_out),
                  "num_effective_tokens": sum(
                      r["num_effective_tokens"] for r in row_out
                  ),
                  "spans": row_out,
              }
          )
      assert (
          len([t for t in token_spans if not t.get("used", False)]) == 0
      )
      return combined_rows
  
imports: import copy
  import functools
  import os
  import tempfile
  import typing
  import sys
  from finetune.base_models import TableRoBERTa
  from finetune.util.metrics import sequences_overlap
  from finetune.scheduler import Scheduler
  from finetune.encoding.input_encoder import BaseEncoder
  from finetune import SequenceLabeler
label_desc: Use image node_id 7 for calling the TableChunker obj's underlying member method code with example usage: obj.combine_row_spans(row_spans, token_spans) and returns: combined_rows, t