class_docstr: Custom Dataset wrapping language modeling sequences.
  
  Each sample will be retrieved by indexing the list of token_ids and their corresponding lengths.
  
  Input:
  ------
      params: `NameSpace` parameters
      data: `List[np.array[int]]
docstr: Too short sequences are simply removed. This could be tuned.
docstr_tok: ["Too","short","sequences","are","simply","removed",".","This","could","be","tuned","."]
code: def remove_empty_sequences(self):
      """
      Too short sequences are simply removed. This could be tuned.
      """
      init_size = len(self)
      indices = self.lengths > 11
      self.token_ids = self.token_ids[indices]
      self.lengths = self.lengths[indices]
      new_size = len(self)
      logger.info(
          f"Remove {init_size - new_size} too short (<=11 tokens) sequences."
      )
  
file_path: H2O/h2o_flexgen/benchmark/third_party/transformers/examples/research_projects/distillation/lm_seqs_dataset.py
imports: import numpy
  import torch
  from torch.utils.data import Dataset
  from utils import logger
label_desc: Use image node_id 6 for calling the LmSeqsDataset obj's underlying member method code with example usage: obj.remove_empty_sequences() without return types