class_docstr: Tasks store dictionaries and provide helpers for loading/iterating over
  Datasets, initializing the Model/Criterion and calculating the loss.
  
  Tasks have limited statefulness. In particular, state that needs to be
  saved to/loaded from checkpoints needs to be stored in the `self.state`
  :class:`StatefulContainer` object. For example::
  
      self.state.add_factory("dictionary", self.load_dictionary)
      print(self.state.dictionary)  # calls self.load_dictionary()
  
  This is necessary so that when loading checkpoints, we can properly
  recreate the task state after initializing the task instance.
class_docstr_tok: ["Tasks","store","dictionaries","and","provide","helpers","for","loading\/iterating","over","Datasets",",","initializing","the","Model\/Criterion","and","calculating","the","loss",".","Tasks","have","limited","statefulness",".","In","particular",",","state","that","needs","to","be","saved","to\/loaded","from","checkpoints","needs","to","be","stored","in","the","`","self.state","`",":","class",":","`","StatefulContainer","`","object",".","For","example",":",":","self.state.add_factory","(","``","dictionary","''",",","self.load_dictionary",")","print","(","self.state.dictionary",")","#","calls","self.load_dictionary","(",")","This","is","necessary","so","that","when","loading","checkpoints",",","we","can","properly","recreate","the","task","state","after","initializing","the","task","instance","."]
docstr: Filter examples that are too large
  
  Args:
      indices (np.array): original array of sample indices
      dataset (~fairseq.data.FairseqDataset): dataset to batch
      max_positions (optional): max sentence length supported by the
          model (default: None).
      ignore_invalid_inputs (bool, optional): don't raise Exception for
          sentences that are too long (default: False).
  Returns:
      np.array: array of filtered sample indices
code: def filter_indices_by_size(
      self,
      indices,
      dataset,
      max_positions=None,
      ignore_invalid_inputs=False,
  ):
      """
      Filter examples that are too large
  
      Args:
          indices (np.array): original array of sample indices
          dataset (~fairseq.data.FairseqDataset): dataset to batch
          max_positions (optional): max sentence length supported by the
              model (default: None).
          ignore_invalid_inputs (bool, optional): don't raise Exception for
              sentences that are too long (default: False).
      Returns:
          np.array: array of filtered sample indices
      """
      indices, ignored = dataset.filter_indices_by_size(
          indices, max_positions
      )
      if len(ignored) > 0:
          if not ignore_invalid_inputs:
              raise Exception(
                  (
                      "Size of sample #{} is invalid (={}) since max_positions={}, "
                      "skip this example with --skip-invalid-size-inputs-valid-test"
                  ).format(
                      ignored[0],
                      dataset.size(ignored[0]),
                      max_positions,
                  )
              )
          logger.warning(
              (
                  "{:,} samples have invalid sizes and will be skipped, "
                  "max_positions={}, first few sample ids={}"
              ).format(len(ignored), max_positions, ignored[:10])
          )
      return indices
  
imports: import logging
  import os
  import warnings
  from argparse import Namespace
  from typing import Any, Callable, Dict, List
  import torch
  from fairseq import search, tokenizer, utils
  from fairseq.logging import metrics
  from fairseq.data import Dictionary, FairseqDataset, data_utils, encoders, iterators
  from fairseq.dataclass import FairseqDataclass
  from fairseq.dataclass.utils import gen_parser_from_dataclass
  from fairseq.optim.amp_optimizer import AMPOptimizer
  from omegaconf import DictConfig
label_desc: Use image node_id 10 for calling the FairseqTask obj's underlying member method code with example usage: obj.filter_indices_by_size(indices, dataset, max_positions, ignore_invalid_inputs) and returns: indices