CycleResearcher: Improving Automated Research via Automated Review
Large Language Models;Automation of Scientific Discovery;AI Scientist
Yes
No ethics review needed.
The framework is primarily designed for machine learning-related research. How do you envision adapting CycleResearcher and CycleReviewer to other scientific fields, such as biology or social sciences, where the nature of research and evaluation criteria might differ significantly?
The paper mentions the potential issue of reward hacking, where the policy model might exploit loopholes in the reward model. Could you elaborate on the specific strategies you are considering to mitigate this issue and ensure that the generated research outputs maintain high academic rigor and novelty?
The introduction of CycleResearcher and CycleReviewer models to automate the entire research process, including literature review, manuscript preparation, peer review, and revision, is highly innovative. This framework mimics the real-world research cycle, enhancing the efficiency and consistency of scientific inquiry.
Performance Improvement: The CycleReviewer model demonstrates a significant improvement in predicting paper scores, outperforming human reviewers by 26.89% in mean absolute error (MAE). This indicates that the model can provide more accurate and consistent evaluations than individual human reviewers.
Quality of Generated Papers: The CycleResearcher model generates papers with an average quality close to human-written preprints, achieving an acceptance rate of 31.07%. This shows that the model can produce high-quality research outputs that are competitive with human-generated content.
Large-Scale Datasets: The development of the Review-5k and Research-8k datasets, which capture the complexities of peer review and research paper generation, provides valuable resources for training and evaluating models in academic paper generation and review.
The paper explores the use of open-source large language models to automate the entire research process, from literature review and manuscript preparation to peer review and revision. The proposed framework includes CycleResearcher, which performs research tasks, and CycleReviewer, which simulates the peer review process. The study demonstrates that CycleReviewer can outperform human reviewers in predicting paper scores, and CycleResearcher can generate papers of quality close to human-written preprints. The models are trained using two new datasets, Review-5k and Research-8k, which capture the complexities of peer review and research paper generation. The results indicate that this approach can significantly enhance the efficiency and quality of scientific research, while also providing ethical safeguards to prevent misuse.
Generalizability Across Domains: The models are primarily designed for machine learning-related research, and their generalizability to other scientific fields remains unexplored. This limitation suggests that the framework might not perform as well in domains outside of machine learning.
Reward Design: The paper highlights the issue of reward definition, where the policy model might exploit loopholes in the reward model to maximize rewards without genuinely improving the quality of the generated research. This behavior could undermine the long-term goal of producing high-quality research outputs.
Complexity of Implementation: Implementing the framework requires significant computational resources and expertise in reinforcement learning and LLMs. This complexity might be a barrier for widespread adoption, especially for smaller research teams or institutions with limited resources.
Yes
No ethics review needed.
See Weaknesses.
Originality: The idea to design both a researcher and a reviewer is novel and interesting. 

Quality: The usage of recent preference optimization methods is a nice technical plus. The work contributes datasets to the direction of scientific peer reviewing, which is a resource that is rather helpful for the field. RL details and how they fit in is nice. 

Clarity: Figures are well-designed and artistically pleasing. Appreciate the various different ways that are used to evaluate the methods (qualitative, ablations, etc.)

Significance: The automation of scientific research and reviewing is a very interesting and timely topic. In particular, due to the massive increase in submissions year-to-year, progress towards the paper's direction is well appreciated.
The paper presents CycleResearcher and CycleReviewer, which is a cohesive system intended to make steps towards automatic scientific discovery. In particular the novelty of the approach lies in the encapsulation of the entire research pipeline from research to review, in order to better model the entire system of research generation and get better outcomes. The authors contribute two datasets for research and review, and use these to train the system of researcher and reviewer, and evaluate them using various methods and metrics.
Originality: N/A

Quality: 
One big issue of the paper is the method in which the authors obtain the "ground truth" review score: "for each submission, we use the average of the other n − 1 reviewers’ scores as an estimator of the true score."
In my opinion (and what feels like a general consensus in the community), it's pretty clear that this isn't the correct approach in determining a ground truth quality of a paper. Different reviewers have different expertises and opinions, and may disagree substantially based on their backgrounds, but this is a positive quality of peer review rather than a negative one. Thus, the metric used to judge the "loss" of a review score can be used to train proxies of reviewers, sure, but it does not make sense to then take the trained system and use the same metric to compare it against actual human reviewers. For instance, if human reviewers vary differently based on their perspectives and CycleReviewer is just doing some "hedge" where most scores are around the median score for all papers, it might achieve better than human performance on the MAE metric that is used in the evaluation. Furthermore, focusing on the score ignores perhaps the more important points of paper reviewing, such as being able to highlight errors in the paper or provide advice for making changes that are adopted in future versions. I think the true objective of reviewing in the paper's cycle paradigm matches these objectives more, although I recognize that they are even harder to quantify. Even so, I think the paper is overclaiming by saying that the lower MAE suggests that "LLMs can surpass expert-level performance in research evaluation".

And since I don't necessarily agree with the evaluation metric for the reviewer, this casts doubt on the results for the CycleResearcher because the CycleReviewer is reviewing the CycleResearcher. Also, since CycleResearcher is optimized on CycleReviewer, then saying that CycleResearcher does better on CycleReviewer than humans or AI scientist doesn't mean much. The qualitative study in section 4.3 is helpful to remove some of these doubts though. 

Clarity: There are a reasonable amount of typos and grammatical errors in the document. For instance, CycleReviewer is replaced with WhizeReviewer in Section 4.1. The title of section 4.1 should be "Experiments on Paper Review Generation", etc. The paper would benefit from a pass over to correct grammatical mistakes in general to make it easier to read. 

Significance: The claims are very catchy that the system can generate better reviews and papers than humans. However, given the questionable-ness of the metric, I think these are discounted to a degree.
Yes
I do not think that the task authors train models for — hallucinating experiment results and writing papers for them — is well motivated. Using models for this purpose will not contribute real knowledge to the scientific field. I think this is dual use technology, if not a completely malicious one. I could imagine the paper could be reframed to center on demonstrating this imminent failure of the reviewing system and raising an alarm, allowing the scientific community to adapt. In current form, the paper is probably net-negative. I think the results from this paper should be known to the broad public, but not in the current framing.
Yes, Potentially harmful insights, methodologies and applications
1. What exactly are the prompts, based on which CycleResearcher generates papers during evaluations?  
2. Why do smaller CycleResearcher models get better scores in the evaluation?  
3. How many samples in automated evaluation?  
4. Please include the average real score of accepted papers given by human ICLR2024 reviewers.  
5. How do you compute the acceptance rates, e.g. one mentioned in line 128?  
6. For human evaluation:  
   1. Please report the N used in best-of-N / rejection sampling.  
   2. Please clarify whether each paper is evaluated by one or several humans.  
   3. How are the human experts chosen?  
   4. What do you mean by saying “excluding formatting considerations” in the assessment, and why is it omitted?
* Training LLMs with reinforcement learning on parts of the AI research process is a novel and significant contribution.
* The paper includes numerous experiments and ablations. The overall methodology is sound (with exceptions, see weaknesses).
* The authors achieve strong results on the metrics they choose. It is somewhat impressive that their system achieved an acceptance rate of 31.07%, similar to ICLR 2024's acceptance rate.
* Authors use open-source models with a large range of scale (from 12B to 123B).
The paper introduces an iterative training framework for automatic generation and review of research papers using open-source LLMs. The core of their approach consists of two main components:
1. CycleResearcher: A policy model that generates the paper, prompted by abstracts of related work.
2. CycleReviewer: A reward model that writes several peer reviews and returns scores according to ICLR criteria.


The authors initialize these models by supervised fine-tuning on scraped conference papers and ICLR reviews. They then improve the CycleResearcher using reinforcement learning (specifically iterative Simple Preference Optimization, SimPO), using CycleReviewer as a reward model.


The paper claims three main contributions:
1. Development of an iterative reinforcement learning framework that mirrors the real-world research-review-revision cycle.
2. Creation of two new datasets: Review-5k. Research-8k
3. Empirical results showing:
   - CycleReviewer produces scores that are closer to averages of multiple human reviewers than scores by individual human reviewers
   - CycleResearcher-12B achieved paper quality scores surpassing preprint level and approaching accepted paper level

The paper implements some ethical safeguards: they train a model to detect papers generated by LLMs they publish; they promise to implement a licensing agreement such that downloading model weights requires sharing institutional affiliations and agreeing not to use models for official peer reviews or submissions without disclosure.
* The writing is overclaiming the extent to which the paper covers the full research process. Authors write that the paper “explores performing the *full* cycle of automated research and review”, however the paper omits crucial part of the process: actually running experiments. Instead, the authors train models to write complete papers purely from abstracts of past work, with completely hallucinated experiment design and results.  
* I do not think that the task authors train models for — hallucinating experiment results and writing papers for them — is well motivated. Using models for this purpose will not contribute real knowledge to the scientific field. I think this is dual use technology, if not a completely malicious one. I could imagine the paper could be reframed to center on demonstrating this imminent failure of the reviewing system and raising an alarm, allowing the scientific community to adapt. In current form, the paper is probably net-negative.  
* Automated evaluation of papers produced by CycleResearcher is hard to trust, since CycleResearcher was trained with RL against the same reward model as used at test-time. Reward model overoptimization (Gao et al, 2022 \- [https://arxiv.org/abs/2210.10760](https://arxiv.org/abs/2210.10760)) should be the expected result of RL, however the authors do not run any experiments to investigate to which extent their evaluation is influenced by this. For example, the authors could train a held-out reward model on a held-out dataset of reviews and then evaluate CycleResearcher on both the reward model used for RL training and this new held-out reward model.  
* The claim that CycleResearcher surpasses the quality of preprint papers and approaches quality of accepted papers is not well supported, due to the concerns about reward model overoptimization mentioned above. Human reviewers rate CycleResearcher’s papers significantly lower (4.8) than the automated reviewer made by the authors (5.36). The authors could have reported the actual historical average score of ICLR2024 accepted papers.   
* I have a number of concerns about the human evaluation procedure.  
  * When the authors evaluate their CycleResearcher with the AI Scientist, they seem to only use rejection sampling (best of N) for CycleResearcher. This is not a fair comparison.  
  * Overall, human evaluation is conducted on a small scale (10 papers total, three human reviewers, 2 methods: this paper and baseline)  
  * I do not think 30min per review (including reading, writing comments & providing scores) is enough\!  
* I think it’s misleading to use the term “revision” for parameters updates of the policy model (Figure 2). The paper refers to this revision as part of the full research process (“Research-Rebuttal-Revision”) but this does not actually involve revision of papers based on reviews.
Yes
**Accountability and Authorship**: If AI systems generate research papers, questions arise regarding authorship and accountability for the content. It's essential to clarify who is responsible for the work produced and how credit should be assigned.

**Quality and Integrity of Research**: The presence of hallucinations and factual inaccuracies in AI-generated papers could undermine the integrity of scientific literature. There is a risk of disseminating false information, which could have downstream effects if other researchers build upon flawed results.

**Misuse of Technology**: The tools developed could be misused to generate large volumes of low-quality or misleading research, potentially cluttering academic discourse and making it harder to identify valuable contributions.

**Impact on the Research Community**: Automation might affect the roles of researchers, peer reviewers, and the collaborative nature of scientific inquiry. There is a need to consider how these technologies will coexist with human efforts and what support structures are necessary to ensure they augment rather than hinder scientific progress.
Yes, Discrimination / bias / fairness concerns Yes, Privacy, security and safety
**Explanation for Performance of Smaller Models**: In Table 3, why does the CycleResearcher-12B model receive the highest acceptance rate compared to the 72B and 123B models? This result is unexpected given that larger models generally have better performance. Could the authors provide an analysis of this outcome, possibly including case studies or error analysis to understand the limitations of larger models in this context?

**Evaluation Stability of CycleReviewer**: What is the temperature setting used for the CycleReviewer during evaluation? Additionally, have the authors experimented with running the CycleReviewer multiple times to assess the variability or deviation in the review scores and feedback? Understanding the stability and consistency of the CycleReviewer is important for gauging its reliability in the automated review process.

**Addressing Hallucinations in Generated Papers**: Given the observed hallucinations and inaccuracies in the sample generated papers (Appendix E), what strategies do the authors propose to mitigate these issues? Are there mechanisms in place to fact-check or verify the content produced by the CycleResearcher before it is submitted for automated review?
**Valuable Datasets**: The introduction of the Review-5k and Research-8k datasets could be highly beneficial to the research community. These datasets provide resources for training and evaluating models in academic paper generation and review, potentially fostering further advancements in automated research tools.

**Innovative Use of Preference Data**: Utilizing preference data to iteratively train the CycleResearcher model is an interesting approach. This method allows the model to improve over multiple iterations, aligning more closely with human standards through reinforcement learning.

**Ethical Safeguards**: The inclusion of a detection model to identify AI-generated papers addresses ethical concerns related to the misuse of automated research tools. By implementing such safeguards, the authors demonstrate a commitment to responsible AI deployment.

**Automation of the Research Lifecycle:** The paper attempts to automate the full research cycle, from idea generation to peer review and revision. This holistic approach is ambitious and, if successful, could significantly impact the efficiency of scientific research.
The authors introduce two core components: CycleResearcher, a policy model that autonomously performs research tasks, and CycleReviewer, a reward model that simulates the peer review process. Experimental results suggest that CycleReviewer can outperform individual human reviewers in scoring consistency, and CycleResearcher shows promise in generating research papers that approach the quality of human-written preprints.
**Quality of Generated Papers**: Upon examining the samples provided in the Appendix (Sections E.1 and E.2), it is evident that the generated papers contain hallucinations and inaccuracies. For instance, in the generated abstracts, there are claims of outperforming state-of-the-art methods without substantial evidence or appropriate citations. This raises concerns about the reliability of the CycleResearcher model in producing high-quality, factual research papers.

**Counterintuitive Results with Model Scaling**: In Table 3 (Section 4.2), the CycleResearcher-12B model achieves a higher acceptance rate than the larger 72B and 123B models. This is counterintuitive, as larger models typically perform better due to increased capacity. The paper does not provide sufficient analysis or explanations for this phenomenon, leaving readers questioning the scalability and efficacy of the approach.

**Insufficient Ethical Considerations**: While the authors mention the implementation of a detection tool for AI-generated papers, the paper lacks a deep exploration of the ethical implications of automating research. Issues such as accountability, potential misuse, and the impact on the scientific community are not thoroughly addressed. A dedicated discussion in the Ethics Considerations section would strengthen the paper.
We propose CycleResearcher and CycleReviewer, open-source LLMs automating the research and review cycle.
@inproceedings{
anonymous2024cycleresearcher,
title={CycleResearcher: Improving Automated Research via Automated Review},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=bjcsVLoHYs},
note={under review}
}
The automation of scientific discovery has been a long-standing goal within the research community, driven by the potential to accelerate knowledge creation. While significant progress has been made using commercial large language models (LLMs) as research assistants or idea generators, the possibility of automating the entire research process with open-source LLMs remains largely unexplored. This paper explores the feasibility of using open-source post-trained LLMs as autonomous agents capable of performing the full cycle of automated research and review, from literature review and manuscript preparation to peer review and paper revision. Our iterative preference training framework consists of CycleResearcher, which conducts research tasks, and CycleReviewer, which simulates the peer review process, providing iterative feedback via reinforcement learning. To train these models, we develop two new datasets, Review-5k and Research-8k, reflecting real-world machine learning research and peer review dynamics. Our results demonstrate that CycleReviewer achieves a 26.89\% improvement in mean absolute error (MAE) over individual human reviewers in predicting paper scores, indicating that LLMs can surpass expert-level performance in research evaluation. In research, the papers generated by the CycleResearcher model achieved a score of 5.36 in simulated peer reviews, surpassing the preprint level of 5.24 from human experts and approaching the accepted paper level of 5.69. This work represents a significant step toward fully automated scientific inquiry, providing ethical safeguards and advancing AI-driven research capabilities.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large Language Models Automation of Scientific Discovery AI Scientist
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/fb730ac4579b99d6a5311030dbebd5502c7e54f5.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/eb06cb5d5ea58ad74de4743a904c97e6551d54f2.zip
CycleResearcher: Improving Automated Research via Automated Review
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission