RedHat: Towards Reducing Hallucination in Essay Critiques with Large Language Models
essay critique generation;large language model;hallucination
Yes
No ethics review needed.
Please try to address my points raised in weaknesses, specifically (a) lack of rooting in literature and comparison between human-written and generated feedback, (b) the discrepancies between claims and empirical evidence of efficacy of your results and (c) whether you could reproduce the results using more LLMs.
Overall this work is intriguing and makes good use of prompt engineering in a clever way. The results seems to indicate some level of success.
The paper introduces a benchmark and corresponding experiments for the task of automatically providing useful free-form feedback for human-written essays. The main objective is to reduce to what the paper refers to as 'hallucinations' - i.e. in this case irrelevant feedback.
To achieve this, the paper introduces a resource called EssayC - essays written in Chinese by students, which are enriched by LLM-generated critiques. The main contribution of the paper seems to be the use of an automatically derived questionnaire aimed at evaluating different aspects pertaining to the quality of the essay, which - together with automatically derived answers - is used to enrich the prompt of LLMs when generating the essay feedback, an approach which the papers calls "RedHat".
I have a few concerns with the paper:

- Firstly, the construction of the evaluation criteria seems rather ad-hoc, GPT4 is prompted for questions, which are then voted upon by human experts in form of GTAs. The results are in no way linked to or rooted in existing rich literature on evaluating essays. I think this is a missed opportuninty to investigate, how well GPT4's question lists align with existing criteria proposed in the literature. Similarly, the generated critiques are not compared to the actual human-written critiques which existed for the collected essays, in fact they were deliberatly removed and disregarded. This again presents a missed opportuninty to further understand the quality of the approach.

- Secondly, for all the prompt engineering, the results are not very impressive. This is down to a few points: Firstly, only a single LLM, chatGLM-9B is evaluated. Secondly, the overall scores in Table 2, pairwise comparisons in Figure 4 and Table 4 show only a small improvement over the chatGLM baseline. In fact given the rather small dataset size, I'm not convinced they are statistically significant. Thirdly, i am not at all convinced by section 5.3 - the Table and the corresponding explaining paragraph (l 428 ff) seem at odds with each other: The paragraph claims that QA-enhanced RedHat methodology increases the overlap (which in itself doesn't mean that the questions are useful), but the table shows the opposite. Indeed the baseline without QA seems to have a higher overlap than the RedHat method.

- Lastly, I am not convinced by the gravity and impact of the contribution - the task of essay feedback generation is rather niche and very application-focussed, only one language is studied and the presented approach amounts to prompt engineering. Generalising the findings would be difficult, as the human studies would need to be reproduced.

Minor concerns include the wording - for example, the decision to use the word "hallucinations" is not very spot-on, "hallucinations" refer to content that is made up in summaries, i don't think what the paper describes as "hallucinations" meets this broad definition. Furthermore, the vocabulary used to outline the contributions is too strong given the actual results - for example l 103 claims to "prove" generalisability, yet I have not found a formal proof. Similarly, l 107 states that human annotators prefer automatically refined essays based on RedHat's feedback, while the majority of the annotators is indifferent to the changes (60% vote tie in Table 4).


Overall, given the concerns outlined below, I would gravitate towards rejection.

My recommendations to improve the paper would be to (a) root the approach more firmly in existing literature e.g. by comparing the question lists with existing literature; (b) increase the scope, by e.g. comparing more models or comparing model results to human-written critiques. The collection of human ratings would also enable the creation of preference data, which could be used to more efficiently align the models (e.g. by RLHF methods) rather than just using SFT. It should also enable to learn a essay quality ranking method, e.g. by regressing the overall/hallu/detail/info scores in table 2. This should greatly increase the impact of the work, as this learned metrics could be used to predict the quality of essays _without human annotations_, thus enabling to generalise the method and reproduce the findings without reproducing the human annotations.
Yes
No ethics review needed.
* There's very little information about EssayC. Including an overview of the dataset (e.g., types and size of data) would be helpful. Additionally, will you open-source the dataset?
* There’s limited information about the baseline methods (e.g., SFT and PT) in the main body of the paper, particularly regarding the training data used. It was challenging to navigate between the main paper and the appendix to find these details.
* You might consider a few-shot approach, where a few examples of essays and their critiques are provided in the prompt. This approach is often effective for tasks like essay scoring for non-native speakers.
* In Section 5.1, the metric "Detailedness" appears to be lower-is-better, while in the objective function (2) it is being maximized. Perhaps renaming it to something like "ambiguity" would clarify its intent.
* The proposed method, RedHat, appears to be grounded in solid assessment practices, with critiques based on concrete practices in answering questions.
* They are working with a non-English (Chinese) dataset and task.
* They provide a clear, position-based analysis highlighting where improvements occurred.
This paper addresses the essay critique generation task using large language models (LLMs) and proposes a technique to reduce hallucination in this task. The authors propose RedHat, a technique to reduce hallucination by incorporating document-level question-answering into the critique generation process. The authors compare the proposed method with others, including post-pretraining (PT) and supervised fine-tuning (SFT), and show that their proposed method does reduce hallucination.
* The proposed method, RedHat, simply adds document-level questions and answers, which may come across as an incremental, somewhat trivial improvement.
* It’s unclear whether the issue the authors address is truly "hallucination" or simply improving task performance in critique generation. Are there types of essay critique generation errors that are not hallucination? A clear definition would be valuable here.
* The paper is quite vague; while the tasks and datasets used are in Chinese, this isn’t clearly stated. Working on non-English tasks and datasets is generally beneficial for diversity in the English-dominated LLM landscape. However, the paper lacks clarity on this and, more importantly, offers very limited insights on whether the findings apply to other datasets, languages, or base LLMs (e.g., GLM). For instance, it would be useful to know if similar techniques could improve essay critique generation in English using models like ChatGPT.
Yes
No ethics review needed.
Q1: From the perspective of the problem defined in Figure 1, if LLM fails to focus on the desired argument or is too focused on the conclusion, is it fit to be called a hallucination?
Q2: I am not sure whether such an open Q-A result is stable. Will it change with each run of model?
1. The authors list definitions of various types of hallucinations in Table 1, which is helpful for clearly defined tasks.
2. The evaluation setting is solid which including both auto and human evaluation.
The background proposed in this paper is to use LLM to evaluate and criticize essay writing. It points out that the efficiency of using LLM is much higher than that of humans. However, LLM has some problems in performing this task, such as (1) Providing suggestions that do not match or are inappropriate to the essay content, and (2) Proposing logical errors that do not exist in the essay. These hallucination seriously affect the usability of LLMs in generating essay reviews.  This paper has two main contributions, a dataset and a new method to mitigate hallucinations. The results of automatic and manual evaluation demonstrate the effectiveness of the work proposed by the authors.
1. From Figure 4, it seems that the evaluation results of GPT are not reliable. This is because people trust the results of artificial experts more. Although the results of untrained human annotators are also random.
2. Why consider ROUGE and BLUE as metrics, as they only consider word overlap. Maybe try BERTScore, BLEURT.
3.The writing structure seems a bit confusing. Chapters 2, 3, and 4 each contain a literature review, plus the work of this paper.
4. It is very good that the authors present a new dataset. But perhaps more annotation process and quality assessment should be provided?
@inproceedings{
anonymous2024redhat,
title={RedHat: Towards Reducing Hallucination in Essay Critiques with Large Language Models},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=IULlNTZZel},
note={under review}
}
Essay critiques refer to the textual assessment of an essay, serving as the basis for the scoring of the essay, and are crucial for the improvements of the essay. Essay critique generation has received increasing attention after the blooming of large language models (LLMs), which show promising potential in writing and critiquing essays. Automatic critique generation can streamline both instructors and reviewers as well as spur LLM advancement in long context generation characterized by essay writing. However, current LLMs suffer from hallucinations when generating essay critiques, which are still under-explored in the community. To facilitate research in reliable essay critique generation, we first define this task with a unified input-output format as well as clear judging criteria. To minimize hallucinations in critique generation, we introduce RedHat, a novel approach that embeds the key information from essays directly into the generation process through document-level question-answering, ensuring critiques stay firmly anchored to the original text. We collected a large-scale, high-quality essay critique dataset called EssayC, annotated by human experts over multiple LLM-generated critiques, from a campus undergraduate essay writing course. We experimented RedHat backboned by commercial and open-sourced LLMs. Results showed that critiques generated by RedHat are preferred by human experts over baseline in 20% of cases on EssayC in detailedness and informativeness, with a decrement of 10% on hallucinations in our judging criteria.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
essay critique generation large language model hallucination
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/9487a1b66891e62205b4384d76ffb1fb7d99c3bf.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/405f608ecd18b80f7124a8af1769a2200e0c202d.zip
RedHat: Towards Reducing Hallucination in Essay Critiques with Large Language Models
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission