Sanitizing LLMs: Retrospective Learning for Self-Correction of Inconsistent Samples via User Preferences
Large language Model;Prompt Based Task;New Downstream Task with Unsupervised Data;Unsupervised Data annotation with User Preference
Yes
No ethics review needed.
- At line 116, referencing a work from 1999 cannot be considered "recently." 
- Additionally, careful attention should be given to the appropriate use of \citet and \citep for citations, as well as ensuring consistent and correct application of \ref and \eqref.
- The results are diverse and demonstrate the effectiveness of the proposed approach.
- The idea behind the proposed ratio and data-centric approach is intuitive.
This paper introduces the Consistent and Inconsistent (CAI) Ratio which serves as an effective evaluation metric for unsupervised data with user preferences and Retrospective Learning(RL) to self-correct the identified inconsistent samples using consistent samples.
- Figure 1 lacks clarity and should present each circle in sequential order. Additionally, the notation in the text does not correspond with the notation in the figure.
- In fact, my primary question throughout the entire paper is why a student model is necessary at all; this aspect does not seem to be addressed in the introduction. The motivation seems to lack clarity and justification. 
- There is a lack of detail regarding the clustering operations.
Yes
No ethics review needed.
Please check the weaknesses.
Strengths: 

1.	For assessing the annotations generated by LLMs or student model, this paper proposes a consistent and inconsistent sample identification and ratio. It can identify the consistent and inconsistent samples. It utilized the predictions from teacher model and student model. If the predictions from these two models are same, then it is a consistent sample. Vise versa.

2.	The retrospective learning method is proposed for self-correction. It includes two modules: divide-and-conquer self-correction and majority voting. With this method, the prediction accuracy is improved because inconsistent samples impact the model performance.
For ensure the model generated labels/responses are aligned with end-user needs, this paper first proposes a consistent and inconsistent ratio for evaluating the performance of LLMs or student generated annotations. The main motivation is the consistent samples have significantly higher accuracy than inconsistent ones. This ratio is used to identify consistent and inconsistent samples. For resolving the self-correction with limited user preference data, this paper also introduces a new method retrospective learning to identify and correct inconsistent samples for minimizing inconsistency and enhancing the accuracy of LLM annotations. The proposed method achieves a better classification accuracy consistently cross datasets with a higher consistent and inconsistent ratio.
Weaknesses:

1.	Although this paper has shown some results on various datasets, it still lacks the comparison with other methods that can handle the similar problem. 

2.	More experiments and explanations about the reliability of the CAI score is needed. And this score is highly depended on the selected teacher model and student model. More explorations and comparisons are needed here.
Yes
No ethics review needed.
Mentioned the question in the weaknesses section above.
The approach proposed in the paper is straightforward, easy to understand and easy to implement which can make people to easily use it.
Authors propose a metric called Consistent and Inconsistent (CAI) Ratio where they identify the ratio of consistant and inconsistent data points. Then they use this metric and propose a method called Retrospective Learning (RL) to make more inconsistent data points consistant which can help boost the accuracy of the generated data.
While the approach proposed in the paper is easy to understand, the paper is not written clearly! Specially the beginning few sections of the paper including the introduction. The reason is that there were many undefined terms in the beginning of the paper that would make it hard to understand the paper in the beginning. The reader might understand the paper only after going through the paper multiple times and decoding the terms. The paper might need significant revision to make it clear. In addition, there are confusing terminology used in the paper like using RL for Retrospective Learning. I would advise changing these acronyms to something that is not widely used to avoid confusion. In addition, the motivation of the work is not super clear to me either. All these points resulted in the paper to not be fully clear and made it hard for me to assess the paper completely. 

Another concern is that authors use the same metric as an evaluation metric (CAI) based on which the mitigation approach is proposed which might not be fair. Although authors use accuracy as a metric as well, it would be good to use other metrics to make the results more robust. Can authors do some human experiments or perhaps use other metrics as well?
@inproceedings{
anonymous2024sanitizing,
title={Sanitizing {LLM}s: Retrospective Learning for Self-Correction of Inconsistent Samples via User Preferences},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=kFALGqLp46},
note={under review}
}
With the advent of large language models (LLMs), using LLMs in conjunction with prompt-based tasks has demonstrated the ability to reduce the high cost and inefficiency of human annotations. Nonetheless, in unsupervised new downstream tasks that require user preferences to align data annotations with expectations, existing evaluation methods for prompt-based tasks become ineffective, especially when ground truth annotations are insufficient or missing. To fill this gap, we propose the novel Consistent and Inconsistent (CAI) Ratio, inspired by our experimental observation that LLMs underperform when the number of inconsistent samples—those with inconsistent predictions across LLMs and the student model—exceeds the number of consistent samples. By estimating the CAI ratio and identifying consistent and inconsistent samples with our proposed CAI identification approach, we aim to minimize inconsistency and enhance the accuracy of LLM-generated annotations for unsupervised data. To achieve this, we introduce Retrospective Learning (RL) with user preference, a data-centric approach that collaborates with the student model and LLMs, using a small number of human annotations as user preferences to resolve inconsistencies in the identified samples. Applied to five domain-specific NLP datasets, our Retrospective Learning approach, leveraging CAI identification, significantly improved the accuracy of LLM-generated responses, with the CAI ratio increasing as the accuracy improved.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large language Model Prompt Based Task New Downstream Task with Unsupervised Data Unsupervised Data annotation with User Preference
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/c6f9c21f43dcc07c7e33491ec0c741fda241ea87.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Sanitizing LLMs: Retrospective Learning for Self-Correction of Inconsistent Samples via User Preferences
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission