class_docstr: Checks for overflow in gradient across parallel process
class_docstr_tok: ["Checks","for","overflow","in","gradient","across","parallel","process"]
code: def has_overflow_serial(self, params):
      for i, p in enumerate(params):
          if p.grad is not None and self._has_inf_or_nan(
              p.grad.data, i
          ):
              return True
      return False
  
imports: from collections.abc import Iterable
  from deepspeed.moe.utils import is_moe_param
  import os
  import psutil
  import gc
  from math import sqrt
  from math import floor
  from bisect import bisect_left
  import torch
  from torch import inf
  from deepspeed import comm
  from deepspeed.utils import groups, logger
  from deepspeed.runtime.constants import PIPE_REPLICATED
  from numpy import prod
label_desc: Use image node_id 4 for calling the CheckOverflow obj's underlying member method code with example usage: obj.has_overflow_serial(params) and returns: False, True