Noisy Data Pruning by Label Distribution Discrimination
data pruning; coreset selection; noise label learning; data centric-ai
Yes
No ethics review needed.
see weaknesses.
- Introduces an innovative NLI-Score for identifying noisy samples by leveraging neighboring sample consistency.  
- Combines feature and label propagation effectively to reduce selection bias in noisy data scenarios.
This paper studied a task combination of label noise and core-set selection. The proposed method, RoP, is a graph-like sample selection strategy that integrates feature propagation and label propagation. The experiment is extensive.
- The discussion on a closely related work, Pr4ReL, requires expansion. (1) Pr4ReL also employs a neighborhood-based strategy for selection and relabeling. Further insights into why RoP outperforms Pr4ReL, along with a more detailed comparison, would strengthen the analysis. (2) Additionally, RoP's performance on CIFAR-10N/100N, as shown in Table 1, does not surpass that of Pr4ReL, which merits further examination.

-  The authors employ CCS for coverage pruning; however, the adaptation of CCS to NLI-Scores is not clearly explained. Additional clarification would help readers understand this approach.

- Experimentation. (1) Additional experimentation on large-scale datasets is recommended, as Mini-WebVision contains only 66k training images. An experiment on Clothing-1M, comparable to the setup in Pr4ReL, would provide more robust validation. (2) The ablation study is limited. In Table 6, the authors should include separate analyses of using feature and label propagation independently.

- Minor typos, such as "retaining" on Line 309, should be corrected for clarity.
Yes
No ethics review needed.
- It’s hard to understand the so-called “intuitive method” in line 70: Since the clean samples are found in the first stage, why do you need to relabel them?
- Are the gradient-based method in line 15 and the loss-based method in line 51 the same thing?
- It proposes a novel two-stage noisy data pruning method.
- It employs novel feature propagation and label propagation to refine neighboring predictions.
In this paper, the authors propose a novel geometry-based noisy data pruning method. It consists of two stages and uses feature propagation and label propagation for reliable neighboring predictions. Experiments demonstrated quite good results.
- The motivation is weak. First, the loss-based method GraNd is not designed and optimized, especially for noisy samples. Thus it is not suitable to choose GradNd to represent the method. Second, it is not convincing to get the conclusion that the geometry-based approach is better than the loss-based approach by simply comparing the two methods. More methods and more noise settings are needed. Moreover, the statement in line 68 applies to all methods, and also can’t lead to the conclusion.
- I recommend the discussion about the noisy data pruning method and sample selection method. It is an important topic that when the pruning rate is limited, the approximation of the noise rate is neglected. Therefore, the result is destined to be sub-optimal.
- From Table 1, Table 2 and Table 6, it seems that the relabeling method is more dominant than selection. I slightly question the motivation to use pruning for mitigating noisy samples since relabeling has solved this problem well.
- The presentation needs to be polished, e.g. the suddenly appeared GraNd in line 52, and some typos, such as 87.9±-1.3 in Table 1 and brackets in line 6.
- The types of noise are insufficient. Symmetric label noise should be taken into experiments.
Yes
No ethics review needed.
In line 250, how to get the ground-truth label of the sample?
- This paper addresses the issue of noisy data pruning, which is crucial in real-world applications.
- The proposed method achieves SOTA results against existing baselines.
This paper studies the problem of noisy data pruning which aims to prune noisy large-scale datasets into concise subsets. The authors first reveal that geometry-based methods exhibit greater resilience to noisy labels compared to gradient-based methods. Then, a discrimination, pruning, and re-labeling method is proposed to conduct noisy data pruning. Specifically, noisy label discrimination is achieved by neighborhood label inconsistency estimation, after feature and label propagation. Then, the pruned set is selected by ensuring coverage on both easy and hard samples. Finally, re-labeling is achieved by SOTA noisy label learning methods. Experiments show the effectiveness of the proposed method.
- The presentation of this paper is poor according to the following aspects:
  - Lacking in-depth analysis of the difference between the proposed method and previous SOTA Pr4ReL, since Pr4ReL also follows a selection and relabeling paradigm and uses neighborhood information. It is important to explain the superiority of the proposed method. Also, authors are encouraged to add noisy data pruning methods in the section of related work.
  - The motivation for feature propagation and label propagation is unclear. Besides, what if directly applying label propagation without feature propagation?
- Line 237 mentions that a model is required to be trained on the noisy label datasets. Since the purpose of dataset pruning is to reduce the training cost, is it reasonable and fair to access the trained model? If so, please refer to some related works.
- The process of pruning and re-labeling actually follows existing works, i.e., a SOTA noisy label learning method SOP+ and Coverage Coreset Sampling strategy, limiting the novelty of the proposed method.
- Typo: In line 309 "retaining retaining"
Yes
No ethics review needed.
See weaknesses.
1. The paper is well-organized, with clear presentations of methodology, experiments, and conclusions that effectively guide the reader through the research.

2. The authors conduct extensive experiments across various datasets, including real-world noisy datasets and synthetic noise datasets, which helps to validate the robustness and applicability of the method.
This paper introduces a novel two-stage robust data pruning method (RoP) aimed at datasets with noisy labels. The first stage identifies clean samples using a Neighborhood Label Inconsistency Score (NLI-Score), followed by a second stage that re-labels the selected samples. RoP employs feature and label propagation to enhance the accuracy of neighboring predictions and uses density-based coverage sampling to balance the number of easy and hard samples across different pruning rates. Extensive experiments demonstrate the effectiveness of RoP on both synthetic noisy datasets and real-world benchmarks.
1. I find this paper completely unoriginal; it merely applies existing techniques from the noisy label learning task to the domain of data pruning. The "FEATURE PROPAGATION" proposed by the authors is present in many works on noisy label learning [1-4]. The authors additionally utilize Equation 5, which involves fusing a sample's own features with those of surrounding samples to improve its own features; however, this may not be very meaningful. Many studies have shown that, in noisy label learning, while model predictions may be misled by noisy labels, the learned features tend to remain reliable. The "LABEL PROPAGATION" proposed by the authors is also found in many works on noisy label learning [5-6]. For the re-labeling part, the authors even explicitly mention using the existing state-of-the-art method, SOP+. 

2. Why are methods related to noisy label learning not compared in the experiments? Many existing methods can be easily adapted to the scenarios presented in this paper.

[1] Multi-Objective Interpolation Training for Robustness to Label Noise. CVPR 2021

[2] Selective-Supervised Contrastive Learning with Noisy Labels. CVPR 2022

[3] RankMatch: Fostering Confidence and Consistency in Learning with Noisy Labels. ICCV 2023

[4] Learning with Neighbor Consistency for Noisy Labels. CVPR 2024

[5] Jo-SRC: A Contrastive Approach for Combating Noisy Labels. CVPR 2021

[6] UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learning. CVPR 2022
In this paper, we introduce a novel robust data pruning method tailored to align with real-world benchmarks, specifically addressing datasets that contain noisy labels.
@inproceedings{
anonymous2024noisy,
title={Noisy Data Pruning by Label Distribution Discrimination},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=6PGT9OJX5N},
note={under review}
}
Data pruning aims to prune large-scale datasets into concise subsets, thereby reducing computational costs during model training.
While a variety of data pruning methods have been proposed, most focus on meticulously curated datasets, and relatively few studies address real-world datasets containing noisy labels. In this paper, we empirically analyze the shortcomings of previous gradient-based methods, revealing that geometry-based methods exhibit greater resilience to noisy labels. Consequently, we propose a novel two-stage noisy data pruning method that incorporates selection and re-labeling processes, which takes into account geometric neighboring information. Specifically, we utilize the distribution divergence between a given label and the predictions of its neighboring samples as an importance metric for data pruning. To ensure reliable neighboring predictions, we employ feature propagation and label propagation to refine these predictions effectively. Furthermore, we utilize re-labeling methods to correct selected subsets and consider the coverage of both easy and hard samples at different pruning rates. Extensive experiments demonstrate the effectiveness of the proposed method, not only on real-world benchmarks but also on synthetic datasets, highlighting its suitability for practical applications with noisy label scenarios.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
data pruning; coreset selection; noise label learning; data centric-ai
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/45268fa15c8c0faecc8a0d3849621376d6d2aae4.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/6ff6aa5ea3464a2e89a969b16fa6a7369d8f3703.pdf
Noisy Data Pruning by Label Distribution Discrimination
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission