class_docstr: Sort TEDLIUM3 samples by target length and batch to max durations.
code: def __init__(self, base_dataset, max_token_limit):
      super().__init__()
      self.base_dataset = base_dataset
  
      idx_target_lengths = [
          (idx, self._target_length(fileid, line))
          for idx, (fileid, line) in enumerate(
              self.base_dataset._filelist
          )
      ]
      idx_target_lengths = [
          (idx, length)
          for idx, length in idx_target_lengths
          if length != -1
      ]
  
      assert len(idx_target_lengths) > 0
  
      idx_target_lengths = sorted(
          idx_target_lengths, key=lambda x: x[1]
      )
  
      assert max_token_limit >= idx_target_lengths[-1][1]
  
      self.batches = batch_by_token_count(
          idx_target_lengths, max_token_limit
      )
  
imports: import os
  from functools import partial
  from typing import List
  import sentencepiece
  import torch
  import torchaudio
  from common import Batch, batch_by_token_count, FunctionalModule, GlobalStatsNormalization, piecewise_linear_log, post_process_hypos, spectrogram_transform, WarmupLR
  from pytorch_lightning import LightningModule
  from torchaudio.models import emformer_rnnt_base, RNNTBeamSearch
label_desc: Use image node_id 1 to create a new CustomDataset object from inherited base classes: torch.utils.data with example: obj = CustomDataset(base_dataset, max_token_limit)