docstr: Advance to a new position in a batch where we have partially processed the batch
  
  If we have previously built a batch of data and made predictions on them, then when we are trying to make
  prediction on later characters in those paragraphs, we can avoid rebuilding the converted data from scratch
  and just (essentially) advance the indices/offsets from where we read converted data in this old batch.
  In this case, eval_offsets index within the old_batch to advance the strings to process.
docstr_tok: ["Advance","to","a","new","position","in","a","batch","where","we","have","partially","processed","the","batch","If","we","have","previously","built","a","batch","of","data","and","made","predictions","on","them",",","then","when","we","are","trying","to","make","prediction","on","later","characters","in","those","paragraphs",",","we","can","avoid","rebuilding","the","converted","data","from","scratch","and","just","(","essentially",")","advance","the","indices\/offsets","from","where","we","read","converted","data","in","this","old","batch",".","In","this","case",",","eval_offsets","index","within","the","old_batch","to","advance","the","strings","to","process","."]
code: def advance_old_batch(self, eval_offsets, old_batch):
      """
      Advance to a new position in a batch where we have partially processed the batch
  
      If we have previously built a batch of data and made predictions on them, then when we are trying to make
      prediction on later characters in those paragraphs, we can avoid rebuilding the converted data from scratch
      and just (essentially) advance the indices/offsets from where we read converted data in this old batch.
      In this case, eval_offsets index within the old_batch to advance the strings to process.
      """
      unkid = self.vocab.unit2id("<UNK>")
      padid = self.vocab.unit2id("<PAD>")
  
      ounits, olabels, ofeatures, oraw = old_batch
      feat_size = ofeatures.shape[-1]
      lens = (ounits != padid).sum(1).tolist()
      pad_len = max(l - i for i, l in zip(eval_offsets, lens))
  
      units = torch.full(
          (len(ounits), pad_len), padid, dtype=torch.int64
      )
      labels = torch.full((len(ounits), pad_len), -1, dtype=torch.int32)
      features = torch.zeros(
          (len(ounits), pad_len, feat_size), dtype=torch.float32
      )
      raw_units = []
  
      for i in range(len(ounits)):
          eval_offsets[i] = min(eval_offsets[i], lens[i])
          units[i, : (lens[i] - eval_offsets[i])] = ounits[
              i, eval_offsets[i] : lens[i]
          ]
          labels[i, : (lens[i] - eval_offsets[i])] = olabels[
              i, eval_offsets[i] : lens[i]
          ]
          features[i, : (lens[i] - eval_offsets[i])] = ofeatures[
              i, eval_offsets[i] : lens[i]
          ]
          raw_units.append(
              oraw[i][eval_offsets[i] : lens[i]]
              + ["<PAD>"] * (pad_len - lens[i] + eval_offsets[i])
          )
  
      return units, labels, features, raw_units
  
imports: from bisect import bisect_right
  from copy import copy
  import numpy
  import random
  import logging
  import re
  import torch
  from torch.utils.data import Dataset
  from .vocab import Vocab
  from stanza.models.common.utils import sort_with_indices, unsort
label_desc: Use image node_id 5 for calling the TokenizationDataset obj's underlying member method code with example usage: obj.advance_old_batch(eval_offsets, old_batch) and returns: units,  labels,  features,  raw_units