Large Language Models have Intrinsic Self-Correction Ability
large language model;self correction;prompt engineering;foundation model
Yes
No ethics review needed.
I wasn’t the most well-positioned to assess the novelty of this work. I took a look through Pan et al. 2024’s survey “Automatically Correcting Large Language Models” paper to help check this, might be useful to reference for other readers?
Solid theoretical argument that runs throughout the paper. Though this isn’t my core area it was quite easy to understand and most claims were plausible.

The experiments are well connected to the theoretical analysis. Neither part feels like an afterthought, which is a bit rare for LLM reasoning papers :)

The discovery of best practices for prompt sets can both be directly applied in engineering work and used to inspire future analysis.
The paper claims to demonstrate the intrinsic self-correction capabilities of LLMs. Motivated from examples where bias in the prompts can interfere with the correction (in particular, by implying there’s a problem with the initial answer), they ask *if they can correct a wrong answer, why can’t an LM get it right the first time?*

They claim that they can’t due to inherent hallucinations but can perform intrinsic SC. They provide some theoretical analysis to demonstrate this, alongside empirical investigations into the impact of temperature, prompt structure, etc.

According to their theoretical analysis, using CoT during the initial answer generation produces a rationale alongside it, which can be attended over in the self-correction step to produce a candidate decision before reasoning for the decision has to be written. However, without the initial chain, generating this decision is effectively relying on another zero-shot correction in line with how the initial answer was (possibly erroneously) generated. In other words, systems where answers are generated by CoT are more robust to self-correction error introduction than ones that don’t use it, and proper self-correction on non-CoT answers essentially just brings performance back up to CoT levels.

This analysis leads to predictions such as hallucination rate increasing with higher temperature leading to worse performance of self-correction. This is confirmed. They also make some claims that feel more dubious to me, like “LLMs are naturally underperforming their intrinsic ability” which feels a little underdetermined to me.
Few, mainly the more philosophical claims I complained about in my summary.
Yes
No ethics review needed.
Please see above.
May be interesting to future research.
This paper investigates the sefl-correction ability of LLMs. In particular, they investigate two factors, i.e. temperature and the bias of prompt. The authors conduct experiments on 6 benchmarks using 4 LLMs. However, the paper is very difficult to read, with less rigor proof. The selection of models is not wide, resulting in less convincing empirical results. Also, I do not see a systematic and coherent motivation to investigate temperature and the bias of the prompt. It is somehow more like assembling these two factors in one paper.
1. The paper is very difficult to read. In particular, the equation part.  Proof 2.1 is very confusing and many explanations are not given, and I cannot understand it with 2 or 3 times reading. For example, in Line 869, "which we denote as correct(A ∈ Q) = λ > k1". What is λ? The portion of correctly answered questions/all questions? And I do not think the probability of that hallucination randomly changing the answer is equal: tokens are of different importance to a sentence (answer), thus, some hallucinated tokens may not change the final answer (this is also related to how the authors define hallucination, but I do not see any justification here). Also, what is h? The portion of hallucinated responses? I also do not understand what the authors mean by h∗(1−λ)/(k−1) answers will be changed from incorrect to correct. Also, why h∗λ> h∗(1−λ)/(k-1)? Line 283, "(a1/T −(1−α)1/T)" should be "α" not "a".
2. The selection of models is limited. More open sourced model should be included for a more comprehensive evaluation.
Yes
No ethics review needed.
1. What are the novel contributions of this paper over previous works on intrinsic self-correction? Given that the paper investigates the topic related to prior work on self-correction (e.g., [Pan et al., 2023](https://arxiv.org/abs/2308.03188)), it would be helpful to clarify how this research extends beyond or differs from earlier findings.
2. Why is some certain experimental results not obviously show the advantage of self-correction? Is it from the method itself, or there are other failure modes?
1. **Insightful Analysis of Self-Correction Mechanisms**: The paper offers a novel perspective by comparing intrinsic self-correction to chain-of-thought and self-verification techniques. This theoretical framing provides a solid foundation for understanding the mechanisms that enable LLMs to self-correct.
2. **Practical Recommendations for Enhanced Model Performance**: By identifying unbiased prompts and zero temperature as key factors for effective self-correction, the authors present valuable insights that are directly applicable to real-world LLM deployment.
The paper investigates the capacity of LLMs to perform self-correction without relying on external sources of information, namely “intrinsic self-correction.” The authors propose that this self-correction process is valuable for enhancing LLM performance and has similar principles as chain-of-thought and self-verification. The study identifies two key factors that influence self-correction: using an unbiased prompt to avoid influencing the model’s initial answer and setting a zero temperature to reduce randomness. Through theoretical analysis and empirical experiments, the paper demonstrates that intrinsic self-correction is achievable across multiple LLMs, providing a foundation for further research in leveraging this capability.
1. **Limited Performance Improvement Even in Ideal Conditions**: The performance gain from self-correction is relatively small, even under the ideal settings proposed (unbiased prompts and zero temperature). As is shown in Table 1 & 2, under many circumstances, the performance gain is less than 2%. This marginal improvement raises questions about the practical impact of the self-correction process and the techniques provided in the paper.
2. **Lack of Significant Theory and Insight**: The paper creatively related the performance of self-correction to chain-of-thought prompting and temperature, but it fails to introduce solid theoretical backbones or experimental insights. 
3. **Limited Model Sizes and Architectures**: The paper only conducted experiments on two API-based models and two open-source models, and it fails to conduct ablation on model sizes (using the same series of models with different sizes) and other model architectures.   This limitation restricts the generalizability of the findings across a broader range of model types.
Yes
No ethics review needed.
N/A
- The problem of self-improvement is of great interest and importance for the research community, particularly given recent debates about whether LLMs possess genuine self-correction abilities.
- The issue of temperature hasn't been studied enough in the context of self-improvement. The authors' attempt to systematically analyze this parameter's influence is valuable.
- The idea to break self-improvement into multiple stages like chain-of-thought and study each component separately (as shown in Section 2.2) is interesting and potentially useful for understanding the mechanism of self-correction.
- **Context: **The paper studies intrinsic self-correction which is LLMs ability to improve **their own** prior decisions (in contrast to **extrinsic** self-correction that uses external feedback of the environment). There are several prior works that cast doubt on the effectiveness of intrinsic self-correction. 

 - **This presented paper:** This work argues that LLMs are indeed capable of intrinsic self-correction capabilities. The paper makes this argument through theoretical analyses and empirical experiments. The paper points out two critical factors for successful self-correction: (1) 
zero temperature and (2) fair prompts.
(1)  Theoretical Foundation Issues: The paper claims to provide theoretical proofs, but the analytical process lacks rigor. For example:
- Eq3 is just wrong, p(A|\tau) should be p(A, R | \tau). Basically, p(X, Y | Z) = p(X | Y, Z) x p(Y|Z)  (minor typo: \tau should be \tau_1). For the same reason, Eq 4 is also incorrect. 
 - Sec 4.1 opens by “the randomness in decision-making diminishes as the temperature decreases”; isn’t this just a trivial statement? In Equation 8, the authors take temperature to extremes to show that answers have more likelihood to flip (which is obvious even without all these equations). But more importantly, this finding has no meaningful connection to self-improvement. Even if it does, it does connect to self-improvement, it is somewhat unrealistic since practical applications rarely use such extreme temperatures.
 - The analysis in Section 5.1 makes an unreasonable assumption that biased prompts cause random answer flipping. This assumption isn't justified, and the subsequent theoretical analysis adds little insight to our understanding.

(2) Overstatement of Results: The authors frequently jump to conclusions without sufficient evidence:
 - In line 194, they cite Table 1 as evidence of "improvement in accuracy after SC", but the difference is merely 0.08% (75.92% to 76.00%).
 - While Table 2 shows statistical significance across multiple datasets, the actual improvements are typically within 1%. This hardly constitutes meaningful improvement.
- In line 311, they make a definitive claim that "GPT-3.5 follows Order 1 of Eq. (5), whereas GPT4 follows Order 2" without providing quantitative evidence for this classification.

(3) Limited Practical Relevance of Temperature Analysis:
 - Figure 2 shows that temperature effects in the practical range (0-0.7) are minimal.
 - The significant effects only appear at temperature values like 1.4, which are rarely used in practice for self-improvement tasks.

-------------

Presentation Issues:
 - Figure 1 would benefit from including key notation (τ1, R1, D, etc.) referenced throughout the paper.
 - Critical prompt examples are relegated to the appendix when they should be in the main text for better understanding of the methodology.
- In Fig1 caption, make it clear what makes your prompt “unbiased”. One might spend a long time reading the prompt and not understanding what they should pay attention to. 

> Feng et al. (2023) argues that CoT increases the effective depth of the circuit by letting the generated outputs repeatedly loop back to the input.

This phrasing is extremely misleading. The arguments of Feng et al. is that with longer generations, Transformers (for **SOME choice of weights**; not necessarily the pre-trained weights) have more expressive capacity to model more complex problems (circuits). However, this argument does not automatically mean that the existing pre-trained LLMs actually do this, which is what is implied by your phrasing. 

Missing references:

There is other works that casts doubt on the abilities of LLMs to self-improve that are missing in your work: 
- Self-[In]Correct: LLMs Struggle with Discriminating Self-Generated Responses, 2024
- The Google paper: LLMs cannot find reasoning errors, but can correct them given the error location, 2024
- ... (there are a few others that I might be missing)
Given that your work directly rebuts the above works, please include appropriate discission for each one (not just Huang et al.) 

Here are several other relevant works that are not cited in your work: 
- Crystal: Introspective reasoners reinforced with self-feedback, 2023
- Generating sequences by learning to self-correct, 2023
- The self-Instruct paper, 2023

Your work is also related to the growing literature on inference-scaling. As the prior work has shown, inference-scaling works. Naturally, one can extend these to data generation, as you do in your work.
Different from the previous perspective, self-correction is an intrinsic ability of an LLM that can be utilized through correct temperature and prompt to enhance its performance.
@inproceedings{
anonymous2024large,
title={Large Language Models have Intrinsic Self-Correction Ability},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=pTyEnkuSQ0},
note={under review}
}
Large language models (LLMs) have attracted significant attention for their exceptional abilities in various natural language processing tasks, but they suffer from hallucinations that will cause performance degradation. One promising solution to improve the LLMs' performance is to ask LLMs to revise their answer after generation, a technique known as self-correction. Among the two types of self-correction, intrinsic self-correction is considered a promising direction because it does not utilize external knowledge. However, recent works doubt the validity of LLM's ability to conduct intrinsic self-correction. In this paper, we present a novel perspective on the intrinsic self-correction capabilities of LLMs through theoretical analyses and empirical experiments. In addition, we identify two critical factors for successful self-correction: zero temperature and fair prompts. Leveraging these factors, we demonstrate that intrinsic self-correction ability is exhibited across multiple existing LLMs. Our findings offer insights into the fundamental theories underlying the self-correction behavior of LLMs and remark on the importance of unbiased prompts and zero temperature settings in harnessing their full potential.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
large language model self correction prompt engineering foundation model
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/77598070301ba9333cc9595aa4adb03634df12af.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Large Language Models have Intrinsic Self-Correction Ability
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission