Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
LLM;Idea Generation;Human Study
Yes
No ethics review needed.
1) RAG pipeline ablation tests: "We keep the top k = 20 papers from each executed function call and stop the action generation when a max of N = 120 papers have been retrieved." , "For retrieval augmentation, we randomly select k = 10 papers from the top-ranked retrieved papers" . Can you please provide some small scale (self-evaluated) tests ablating these parameters to ensure their optimality?

2) Researcher Background: I understand that the researcher familiarity numbers are provided for idea generation and reviewing processes, however it is slightly different from expertise. So, a better way to ensure the robustness of results is to look at the distribution of the expertise of the reviewers. 

3) Interestingness: Can you please clarify the meaning of the term? Is it that the insight behind the proposed idea a non-trivial one or is it that the human evaluating it finds it personally interesting?
1) Extensive human evaluation with over 100 NLP researchers
2) Great care taken to remove confounders through style normalization, review form, grant submission form for research ideas etc. 
3) Interesting insights like lack of ability of LLMs to generate diverse research ideas.
This paper evaluates if LLM based RAG agents are capable of generating novel research ideas for prompting techniques by comparing them with expert researchers. The key contributions of their work are 

1) Evaluating the capabilities of LLMs to generating research ideas and analyzing their strengths (novelty) and weaknesses (feasibility and diversity)
2) They focus on removing confounders by conducting a large scale human evaluations and controlling experimental setting
1) Not strong enough motivation for restricting to prompting based research. This could have been a confounder since we did we are not sure about the ability of LLMs to generate research where the solutions are more complex. 

2) Usefulness: The paper does not talk about the usefulness of the generated ideas research ideas, which is more important than novelty. Research ideas are born out of deep insights into the limitations of previous research. This paper neither gives the discussion/limitations section of research papers to the LM as context nor does it do any analysis of the usefulness of the generated ideas. Using abstracts and paper titles sounds like an anti-causal way of generating ideas which I think could affect the usefulness.
Yes
No ethics review needed.
- The authors used reviews from Openreview, are there any licensing issues?
* Thorough Study Design: This study is detailed and carefully planned, taking a year to complete and requiring significant resources. It includes human expert baselines and a clear evaluation process, with strong statistical methods that make the results reliable.

* Key Conclusion: The study makes an important finding about the differences between AI-generated and human-generated research ideas, adding to our understanding of AI's potential in coming up with research ideas. — more discussions on the absolute quality, in addition to the comparison between AI and human, would be more apprieciated (e.g. are the quality of both ideas bad).

* Interesting Findings: The study also reveals some surprising insights, such as AI-generated ideas becoming repetitive as more ideas are created, and that current AI systems aren’t yet good at reliably evaluating research ideas
- Study: This paper presents a novel study comparing research ideas generated by Large Language Models (LLMs) with those crafted by expert NLP researchers.
- Methodology: The study involves hired expert researchers who generate ideas and conduct blind reviews of both human- and AI-generated ideas.
- Conclusion: This work is the first to conclude that AI-generated ideas are judged as significantly more novel than those from human experts (p < 0.05).
* While this work is interesting and presents important conclusions, it does not center on machine learning methodology or technical analysis. Although it is relevant and intriguing for the ML community, the paper leans more towards evaluating scientific idea generation, aligning more with applications rather than with core ML research.
* The study is limited by its exclusive focus on prompting-based NLP research, which restricts the generalizability of its findings to other fields or other ML directions.
* Although the authors worked to standardize evaluation criteria, research ideation remains inherently subjective, as reflected in the study's inter-reviewer agreement of 56.1%
Yes
No ethics review needed.
1. Could you provide the specific prompt used for the Agent to generate ideas?
2. Observing that some researchers in Table 3 and Table 4 have a familiarity level of 1 with the domain (which could be considered as low-quality annotation), what are the experimental results in Table 5 after removing this data?
1. The results are quality assured due to significant investment. The author recruited a total of 138 excellent natural language processing researchers and verified their quality through Google Scholar. On average, each idea took the researchers 5.5 hours to generate, and there was a reward of $300. As for the evaluators, each evaluation opinion had a reward of $25. This incentive mechanism, to a certain extent, ensures the quality of idea generation and evaluation.
2. It is relatively fair. By standardizing the topics and providing human experts and large language models with the same writing templates, and finally passing both outputs through the same style transformation module for rewriting, the consistency of style between the two is ensured, minimizing human bias towards writing style as much as possible.
3. It is very meaningful. This large-scale manual evaluation verifies the ability of large language models to generate scientific ideas comparable to those of human experts, providing motivation for the development of subsequent work.
They constructed a meticulously controlled experiment comparing human-generated and LLM-generated ideas, thus overcoming the sample size and baseline issues present in previous small-scale evaluation studies. The study recruited 138 outstanding natural language processing researchers to generate human baseline ideas and conduct blind reviews (49 were responsible for writing ideas, and 79 for blind review). To reduce the impact of confounding factors, the study strictly controlled the style of the ideas and standardized the topic distribution between human and LLM-generated ideas. After a year-long high-cost evaluation, the authors provided a human expert baseline and a standardized evaluation protocol, laying a foundation for future related research.

In nearly 300 reviews, the authors found that under multiple hypothesis tests and different statistical tests, AI-generated ideas were considered more novel than those of human experts, but less feasible. In addition to evaluating the ideas, the study also analyzed the LLM agent, revealing its limitations and unresolved issues. Despite high hopes for expanding LLMs' reasoning abilities, the authors found that they still show deficiencies in the diversity of idea generation, and the results of human consistency analysis indicate that LLMs cannot yet serve as reliable evaluators of scientific idea quality.

In summary, the contributions of the paper are as follows:

1. Standardized some evaluation criteria for scientific ideas generated using large language models while demonstrating the limitations of LLMs as judges in evaluating scientific ideas.
2. Conducted a blind review evaluation of the quality of scientific ideas generated by LLMs through the recruitment of a large number of scientists, verifying both the potential and shortcomings of LLMs in the field of scientific research.
1. The author concluded that ideas generated by large language models lack diversity. However, using the same prompts, especially longer prompts, it is easy for large models to produce similar responses. Therefore, the lack of diversity may not be solely due to the models themselves; it could also be due to a lack of diversity in the prompts. The author needs to test more prompts to sufficiently reach such a conclusion for this experiment.
2. The paper only verifies the generation quality of the author's simple implementation of an agent, lacking evaluation against other baselines.
3. The research is limited to the NLP field and lacks studies in other domains.
Yes
No ethics review needed.
See above.
1. The topic is interesting, as employing LLMs to generate research ideas holds practical value for researchers. Additionally, the findings offer valuable insights that can help researchers determine whether to adopt LLMs as part of their research toolkit.

2. The study’s experimental design is well-structured, effectively controlling for potential confounding factors. Through the use of blind reviews and standardized styles and topics, the authors achieve a fair and balanced comparison between ideas generated by humans and those by LLMs.

3. This paper incorporates a large-scale human evaluation, involving over 100 expert NLP researchers to create and assess research ideas. This approach provides a robust evaluation of the ideas across multiple metrics.
This paper investigates the potential of LLMs to autonomously generate innovative research ideas comparable to those created by human experts. 
In a controlled study, over 100 NLP researchers evaluated ideas generated by both humans and an LLM agent across metrics like novelty, excitement, feasibility, and effectiveness. The findings reveal that LLM-generated ideas were rated significantly more novel than human ideas, though slightly lower on feasibility.

The authors addressed multiple experimental challenges, including ensuring a balanced comparison between human and LLM ideas by standardizing topics and using blind reviews. 
They also tested the robustness of results through various statistical methods. The LLM utilized a retrieval-augmented generation approach and scaled inference techniques to produce numerous ideas, from which a ranking mechanism selected the best ones. Despite the promising novelty scores, the study identifies several limitations of LLMs in research ideation, particularly a lack of diversity in ideas and challenges with reliable self-evaluation.

The study highlights open questions in designing research agents and emphasizes the potential and current limitations of LLMs in academic innovation.
1. As noted in L434-L435, I appreciate that the authors acknowledge the subjectivity involved in reviewing ideas rather than executed papers. However, it would still be valuable to compare LLM-generated ideas with those from published papers (as ground truth), as the latter have been deemed at least ``feasible''.

2. According to L175-L179, the LLM agent was created with a minimalist approach, incorporating only paper retrieval, idea generation, and ranking. However, in real-world research, high-quality ideas often benefit from iterative refinement and prototyping, which are essential for improving clarity and feasibility. As such, I view this paper as an initial investigation into whether LLMs, in their current, straightforward form, can generate high-quality research ideas. The limitations observed, such as the lower feasibility of LLM-generated ideas, could potentially be addressed by a more sophisticated agent framework that includes stages for refining and prototyping ideas.

3. I would not consider this as a major weakness, but given the insights provided, it would be beneficial if the authors offered best practices or recommendations for using LLMs effectively to generate research ideas.
@inproceedings{
anonymous2024can,
title={Can {LLM}s Generate Novel Research Ideas? A Large-Scale Human Study with 100+ {NLP} Researchers},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=M23dTGWCZy},
note={under review}
}
Recent advancements in large language models (LLMs) have sparked optimism about their potential to accelerate scientific discovery, with a growing number of works proposing research agents that autonomously generate and validate new ideas. Despite this, no evaluations have shown that LLM systems can take the very first step of producing novel, expert-level ideas, let alone perform the entire research process. We address this by establishing an experimental design that evaluates research idea generation while controlling for confounders and performs the first comparison between expert NLP researchers and an LLM ideation agent. By recruiting over 100 NLP researchers to write novel ideas and blind reviews of both LLM and human ideas, we obtain the first statistically significant conclusion on current LLM capabilities for research ideation: we find LLM-generated ideas are judged as more novel (p < 0.05) than human expert ideas while being judged slightly weaker on feasibility. Studying our agent baselines closely, we identify open problems in building and evaluating research agents, including failures of LLM self-evaluation and their lack of diversity in generation.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
LLM Idea Generation Human Study
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/8f309f018d9906bc45eb7b51afef438095922ef8.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission