class_docstr: Class for finding label issues in big datasets where memory becomes a problem for other cleanlab methods.
  Only create one such object per dataset and do not try to use the same ``LabelInspector`` across 2 datasets.
  For efficiency, this class does little input checking.
  You can first run :py:func:`filter.find_label_issues <cleanlab.filter.find_label_issues>`
  on a small subset of your data to verify your inputs are properly formatted.
  Do NOT modify any of the attributes of this class yourself!
  Multi-label classification is not supported by this class, it is only for multi-class classification.
  
  The recommended usage demonstrated in the examples script below involves two passes over your data:
  one pass to compute `confident_thresholds`, another to evaluate each label.
  To maximize efficiency, try to use the largest batch_size your memory allows.
  To reduce runtime further, you can run the first pass on a subset of your dataset
  as long as it contains enough data from each class to estimate `confident_thresholds` accurately.
  
  In the examples script below:
  - `labels` is a (big) 1D ``np.ndarray`` of class labels represented as integers in ``0,1,...,K-1``.
  - ``pred_probs`` = is a (big) 2D ``np.ndarray`` of predicted class probabilities,
  where each row is an example, each column represents a class.
  
  `labels` and `pred_probs` can be stored in a file instead where you load chunks of them at a time.
  Methods to load arrays in chunks include: ``np.load(...,mmap_mode='r')``, ``numpy.memmap()``,
  HDF5 or Zarr files, see: https://pythonspeed.com/articles/mmap-vs-zarr-hdf5/
  
  Examples
  --------
  >>> n = len(labels)
  >>> batch_size = 10000  # you can change this in between batches, set as big as your RAM allows
  >>> lab = LabelInspector(num_class = pred_probs.shape[1])
  >>> # First compute confident thresholds (for faster results, can also do this on a random subset of your data):
  >>> i = 0
  >>> while i < n:
  >>>     end_index = i + batch_size
  >>>     labels_batch = labels[i:end_index]
  >>>     pred_probs_batch = pred_probs[i:end_index,:]
  >>>     i = end_index
  >>>     lab.update_confident_thresholds(labels_batch, pred_probs_batch)
  >>> # See what we calculated:
  >>> confident_thresholds = lab.get_confident_thresholds()
  >>> # Evaluate the quality of the labels (run this on full dataset you want to evaluate):
  >>> i = 0
  >>> while i < n:
  >>>     end_index = i + batch_size
  >>>     labels_batch = labels[i:end_index]
  >>>     pred_probs_batch = pred_probs[i:end_index,:]
  >>>     i = end_index
  >>>     batch_results = lab.score_label_quality(labels_batch, pred_probs_batch)
  >>> # Indices of examples with label issues, sorted by label quality score (most severe to least severe):
  >>> indices_of_examples_with_issues = lab.get_label_issues()
  >>> # If your `pred_probs` and `labels` are arrays already in memory,
  >>> # then you can use this shortcut for all of the above:
  >>> indices_of_examples_with_issues = find_label_issues_batched(labels, pred_probs, batch_size=10000)
  
  Parameters
  ----------
  num_class : int
    The number of classes in your multi-class classification task.
  
  store_results : bool, optional
    Whether this object will store all label quality scores, a 1D array of shape ``(N,)``
    where ``N`` is the total number of examples in your dataset.
    Set this to False if you encounter memory problems even for small batch sizes (~1000).
    If ``False``, you can still identify the label issues yourself by aggregating
    the label quality scores for each batch, sorting them across all batches, and returning the top ``T`` indices
    with ``T = self.get_num_issues()``.
  
  verbose : bool, optional
    Whether to suppress print statements or not.
  
  n_jobs: int, optional
    Number of processes for multiprocessing (default value = 1). Only used on Linux.
    If `n_jobs=None`, will use either the number of: physical cores if psutil is installed, or logical cores otherwise.
  
  quality_score_kwargs : dict, optional
    Keyword arguments to pass into :py:func:`rank.get_label_quality_scores <cleanlab.rank.get_label_quality_scores>`.
  
  num_issue_kwargs : dict, optional
    Keyword arguments to :py:func:`count.num_label_issues <cleanlab.count.num_label_issues>`
    to control estimation of the number of label issues.
    The only supported kwarg here for now is: `estimation_method`.
docstr: Update the estimate of num_label_issues stored in this class using a new batch of data.
  Kwargs are ignored here for now (included for forwards compatibility).
  Instead of being specified here, `estimation_method` should be declared when this class is initialized.
docstr_tok: ["Update","the","estimate","of","num_label_issues","stored","in","this","class","using","a","new","batch","of","data",".","Kwargs","are","ignored","here","for","now","(","included","for","forwards","compatibility",")",".","Instead","of","being","specified","here",",","`","estimation_method","`","should","be","declared","when","this","class","is","initialized","."]
code: def _update_num_label_issues(
      self,
      labels: LabelLike,
      pred_probs: np.ndarray,
      **kwargs,
  ):
      """
      Update the estimate of num_label_issues stored in this class using a new batch of data.
      Kwargs are ignored here for now (included for forwards compatibility).
      Instead of being specified here, `estimation_method` should be declared when this class is initialized.
      """
  
      # whether to match the output of count.num_label_issues exactly
      # default is False, which gives significant speedup on large batches
      # and empirically matches num_label_issues even on input sizes of
      # 1M x 10k
      thorough = False
      if self.examples_processed_thresh < 1:
          raise ValueError(
              "Have not computed any confident_thresholds yet. Call `update_confident_thresholds()` first."
          )
  
      if self.n_jobs == 1:
          adj_confident_thresholds = (
              self.confident_thresholds - FLOATING_POINT_COMPARISON
          )
          pred_class = np.argmax(pred_probs, axis=1)
          batch_size = len(labels)
          if thorough:
              # add margin for floating point comparison operations:
              pred_gt_thresholds = (
                  pred_probs >= adj_confident_thresholds
              )
              max_ind = np.argmax(
                  pred_probs * pred_gt_thresholds, axis=1
              )
              if not self.off_diagonal_calibrated:
                  mask = (max_ind != labels) & (pred_class != labels)
              else:
                  # calibrated
                  # should we change to above?
                  mask = pred_class != labels
          else:
              max_ind = pred_class
              mask = pred_class != labels
  
          if not self.off_diagonal_calibrated:
              prune_count_batch = np.sum(
                  (
                      pred_probs[np.arange(batch_size), max_ind]
                      >= adj_confident_thresholds[max_ind]
                  )
                  & mask
              )
              self.prune_count += prune_count_batch
          else:  # calibrated
              self.class_counts += value_counts_fill_missing_classes(
                  labels, num_classes=self.num_class
              )
              to_increment = (
                  pred_probs[np.arange(batch_size), max_ind]
                  >= adj_confident_thresholds[max_ind]
              )
              for class_label in range(self.num_class):
                  labels_equal_to_class = labels == class_label
                  self.normalization[class_label] += np.sum(
                      labels_equal_to_class & to_increment
                  )
                  self.prune_counts[class_label] += np.sum(
                      labels_equal_to_class
                      & to_increment
                      & (max_ind != labels)
                      # & (pred_class != labels)
                      # This is not applied in num_label_issues(..., estimation_method="off_diagonal_custom"). Do we want to add it?
                  )
      else:  # multiprocessing implementation
          global adj_confident_thresholds_shared
          adj_confident_thresholds_shared = (
              self.confident_thresholds - FLOATING_POINT_COMPARISON
          )
  
          global labels_shared, pred_probs_shared
          labels_shared = labels
          pred_probs_shared = pred_probs
  
          # good values for this are ~1000-10000 in benchmarks where pred_probs has 1B entries:
          processes = 5000
          if len(labels) <= processes:
              chunksize = 1
          else:
              chunksize = len(labels) // processes
          inds = split_arr(np.arange(len(labels)), chunksize)
  
          if thorough:
              use_thorough = np.ones(len(inds), dtype=bool)
          else:
              use_thorough = np.zeros(len(inds), dtype=bool)
          args = zip(inds, use_thorough)
          with mp.Pool(self.n_jobs) as pool:
              if not self.off_diagonal_calibrated:
                  prune_count_batch = np.sum(
                      np.asarray(
                          list(
                              pool.imap_unordered(
                                  _compute_num_issues, args
                              )
                          )
                      )
                  )
                  self.prune_count += prune_count_batch
              else:
                  results = list(
                      pool.imap_unordered(
                          _compute_num_issues_calibrated, args
                      )
                  )
                  for result in results:
                      class_label = result[0]
                      self.class_counts[class_label] += 1
                      self.normalization[class_label] += result[1]
                      self.prune_counts[class_label] += result[2]
  
imports: import numpy
  from typing import Optional, List, Tuple, Any
  from cleanlab.count import get_confident_thresholds, _reduce_issues
  from cleanlab.rank import find_top_issues, _compute_label_quality_scores
  from cleanlab.typing import LabelLike
  from cleanlab.internal.util import value_counts_fill_missing_classes
  from cleanlab.internal.constants import CONFIDENT_THRESHOLDS_LOWER_BOUND, FLOATING_POINT_COMPARISON, CLIPPING_LOWER_BOUND
  import platform
  import multiprocessing
label_desc: Use image node_id 8 for calling the LabelInspector obj's underlying member method code with example usage: obj._update_num_label_issues(labels, pred_probs) without return types