COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement
autoregressive large language modeling;decoding;iterative refinement
Yes
No ethics review needed.
- If the way to generate tokens in the first step is different from that in the process of iterative refinements? Are there any better methods to generate draft tokens.
- This paper is well-writen and easy to follow. 
- The performance on logical reasoning tasks are good.
This paper proposes Context-Wise Order-Agnostic Language Modeling (COrAL), which incorporates iterative refinement directly into the LLM architecture while maintaining computational efficiency. Empirical evaluations on reasoning tasks demonstrate that COrAL improves performance and inference speed, and results on code generation indicate a drop in pass rates due to inconsistencies in order-agnostic outputs, highlighting the inherent quality–speed trade-off.
- I think this paper is similar to the other type of works, i.e., speculative decoding, what the difference between them? 
- The noverty is limited, since the specific ways for iterative refinements, the training methods to learn correction, are borrowed from previous works. 
- The significant one: this method seems to only work in specific tasks, the logical reasoning tasks in this paper. However, we always focus on the generalization of current language models, i.e., the competitive on a wide range of tasks.
Yes
No ethics review needed.
The clarity of the paper is good, it's easy for people to follow generally. I don't have further questions.
- Improved Efficiency and Performance:  COrAL’s order-agnostic framework allows simultaneous forward and backward processing, significantly reducing inference latency compared to traditional autoregressive models. Compared to the ablated baselines, empirical results on datasets like GSM8K and LogiQA demonstrate notable accuracy gains, confirming the model’s effectiveness in complex reasoning tasks.
- Scalably Adaptable from Existing Models: By using context-wise modeling and target-aware positional encoding, COrAL manages to enhance dependency capture without substantially increasing computational resources, making it feasible for deployment in large-scale applications, even with existing large language models with only minor adaptation.
The paper proposes COrAL(Context-Wise Order-Agnostic Language Modeling), a novel architecture for language modeling that enhances efficiency in iterative refinement, aiming to reduce inference latency in large language models (LLMs). Traditional autoregressive models, which generate text sequentially, struggle with efficiency due to the natural linear time complexity in inference. COrAL incorporates iterative refinement directly into the model, allowing multi-token generation and backward reconstruction within manageable context windows. This order-agnostic approach enables simultaneous forward and backward decoding within sliding context windows, effectively accelerating inference and improving performance on reasoning tasks. Empirical tests show significant improvements in both accuracy and inference speed, demonstrating COrAL's promise in capturing diverse token dependencies without the high latency typical of AR models. However, challenges remain, such as reduced performance in code generation due to output consistency issues, indicating areas for further refinement.
- Lack of survey of some (maybe kind of obsolete yet important) existing methods: This method resembles Scheduled Sampling in multiple aspects, yet it severely lacks the acknowledgement of this method (no citation nor even mentioning). It shares many ideas and practices with SS, necessitating a deeper analysis on the connection and differences between the method. For example, I'd recommend the authors to emphasize the capability of the proposed method on semi-parallel, refinitive generation, whereas SS was originally only proposed for improvements of performance in sequential generation. 
- Lack of deeper discussion on the theoretical insights: I appreciate the authors' awesome work in presenting and delivering the empirical results, but I presume it would appeal the community more if some insightful conclusions can be presented alongside the experiment observations.
Yes
No ethics review needed.
It would be really interesting to check how much performance is lost by starting from a pretrained model as compared to full training a method employing coral from scratch. Do you think that some performance is left on the table because you start from a pretrained model? 

In the main result part, to increase my rating I would like to see a comparison to other interative refinement methods that have a similar computational cost as the w/o multi-token prediction variant of the proposed method and also a more detailed description of the autoregressive baseline.

Suggestion: Maybe it would be a good idea to incorporate an application in which this method shines. E.g., by looking into domains that can benefit from the order-agnostic aspect such as protein language modelling.
* the authors propose an interesting paradigm and show that it has promise for reducing computational cost and enhancing performance in certain settings
* the method is applicable to autoregressive pretrained language models and seems to improve their performance in certain settings
* the authors provide a quite extensive ablation study for their method
* the paper contains some beautiful figures such as figure (2) and (3). Even though Figure (2) is a little bit unclear to me. Why are there seemingly different offsets for the refinements and why is there not much visual seperation inbetween forward prediction and refinement?
The authors introduce a novel decoding strategy combining autoregressive modelling with ROBERTA-like order agnostic refinement. Given a partial sequence, they predict multiple tokens ahead, which they subsequently refine using ROBERTA-like denoising autoencoder. The authors see performance improvements on GSM8K and LogiQA and poor performance on code generation.
* pseudo-code for Algorithm 1 is provided without walking through the pseudo-code
* in the experimental section the baselines are not described in enough detail, just AR. the proposed method requires finetuning, are the AR baselines also finetuned on the tasks?
* the by-far-best performance is achieved using the w\o multi-head prediction ablation, which is not the proposed method and thus weird. I assume this variant suffers from increased computational cost compared to the proposed method. It would be interesting to compare this ablation with a method from the related work that has a similar computational cost.
* comparison to refinement methods from the related work is missing
* a somewhat non-standard notation for expected values is used. their subscripts seem to be used much like in summations, but usually subscripts at an expected value are used to indicate over which distribution the expectation is taken: e.g., equation (1) and equation (3)
Yes
No ethics review needed.
In eq.8 entropy is always positive, so -H(x) is always negative and exp(-H(x)) is always less than 1. So min(a,a*exp(-H(x))) is always a*exp(-H(x)).
1. The topic of the paper is interesting, transformer-based model do have the problem of slow decoding speed. 

2. It make a good balance between the speed and the performance.
The authors propose a new decoding method, called CORAL, which can speed up the decoding process and maintain (or upraise) the performance of the model in some tasks. CORAL has 2 parts: prediction and verification. The experiment shows that the verification part can help the model to generate more accurate results. CORAL also designed a strategy named "multi-forward" to speed up the decoding process (although it may hurt the performance). The result shows that the CORAL is useful in math problems but is useless in the code generation task.
1. The improvement of the CORAL is not generalizable enough. It only works well in the some math/logic problems but not in the code generation task.

2. Although the speed of the decoding process is improved, it needs to use more GPU memory (and "waste" some computation because of verification and multi-forward) to achieve this. So it is not friendly to equipment that most people use.
@inproceedings{
anonymous2024coral,
title={{CO}r{AL}: Order-Agnostic Language Modeling for Efficient Iterative Refinement},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=0JjsZC0w8x},
note={under review}
}
Iterative refinement has emerged as an effective paradigm for enhancing the capabilities of large language models (LLMs) on complex tasks. However, existing approaches typically implement iterative refinement at the application or prompting level, relying on autoregressive (AR) modeling. The sequential token generation in AR models can lead to high inference latency. 
To overcome these challenges, we propose **C**ontext-Wise **Or**der-**A**gnostic **L**anguage Modeling (COrAL), which incorporates iterative refinement directly into the LLM architecture while maintaining computational efficiency. Our approach models multiple token dependencies within manageable context windows, enabling the model to perform iterative refinement internally during the generation process. Leveraging the order-agnostic nature of COrAL, we introduce sliding blockwise order-agnostic decoding, which performs multi-token forward prediction and backward reconstruction within context windows. This allows the model to iteratively refine its outputs in parallel in the sliding block, effectively capturing diverse dependencies without the high inference cost of sequential generation.
Empirical evaluations on reasoning tasks demonstrate that COrAL improves performance and inference speed, respectively, achieving absolute accuracy gains of $4.6$\% on GSM8K and $4.0$\% on LogiQA, along with inference speedups of up to $3.9\times$ over next-token baselines. Preliminary results on code generation indicate a drop in pass rates due to inconsistencies in order-agnostic outputs, highlighting the inherent quality--speed trade-off.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
autoregressive large language modeling decoding iterative refinement
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/76f3ec579890db3963854a91814f12af10b81768.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/c78173aac3e45206bf8b30d567efe1d5403585b5.zip
COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission