Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversity
Large language model;molecular generative model;drug discovery
Yes
No ethics review needed.
See weakness
1.This paper addresses an important need in molecular generation by developing a method that enhances structural diversity, a key factor in drug discovery. 
2. The two-stage fine-tuning approach, which combines supervised learning with reinforcement learning (RL) for diversity maximization well-designed solution. 
3. Through empirical testing, the paper shows that its approach outperforms both current LLM-based molecular generators and advanced decoding schemes, highlighting the practical advantages of its method for achieving high-quality, diverse outputs.
4. The proposed method is adaptable and could be extended to other domains requiring diversity in generated outputs, such as protein design or materials discovery. This versatility increases the impact of the work beyond just molecular generation.
This paper proposes a two-stage fine-tuning approach to address the challenge of generating diverse molecules: (1) supervised fine-tuning to enable LLMs to autoregressively generate molecules in a sequence, and (2) reinforcement learning to enhance structural diversity among the generated molecules.
The contributions of this work include introducing an innovative method that combines supervised learning and reinforcement learning to generate structurally diverse molecules. Additionally, the paper demonstrates that this approach surpasses existing LLMs and traditional decoding methods in producing high-quality, diverse molecular structures.
1. The two-stage fine-tuning approach which is highlighted in this paper is not that novel.
2. Although the model is tested on standard datasets and metrics for structural diversity, it lacks experimental or real-world validation to demonstrate the generated molecules’ practical utility in drug discovery.
3. The proposed method’s multi-stage fine-tuning approach may require extensive hyperparameter tuning and setup, which can hinder reproducibility and adoption among researchers.
4. The paper claim "the first to explore the use of LLMs for generating diverse molecule" and "the first propose a fine-tuning approach for LLMs to generate diverse solution", which is overstate.
Yes
No ethics review needed.
1. In Table 3, it is unclear why the comparison is made when the number of generated molecules is different. Additionally, all relevant metrics should be included in the comparison to provide a comprehensive assessment.

2. In Table 4, the authors should specify which large language model (LLM) was used as the basis for fine-tuning and reinforcement learning in the compared methods. Furthermore, when comparing Tables 2 and 4, the performance improvement of the Div-SFT approach appears to be minimal.
This paper identifies a limitation in the molecular generation capabilities of LLMs and proposes a diversity enhancement strategy based on autoregressive generation and reinforcement learning.
This paper investigates the limitations of current large language models (LLMs) in generating structurally diverse molecules and proposes a novel two-stage fine-tuning approach to address this challenge. Specifically, the authors apply supervised fine-tuning to enable LLMs to generate a sequence of molecules and subsequently leverage reinforcement learning to enhance structural diversity. Experimental results demonstrate that the proposed method improves molecular diversity compared to existing decoding strategies.
1. The method directly adopts LLMs with reinforcement learning. Thus, the technical novelty of the method is limited.
2. To help readers understand the impact of the approach, the authors could provide additional explanations for the diversity metrics or introduce more similarity evaluation methods.
Yes
No ethics review needed.
1) Can you improve the explanation of NCircles? I think I get it, but given it's a relatively new method I yet don't have an intuition for it like I do for things like T-SNE or UMAP. What kinds of hyperparameters does it have? What are the conditions for two nodes to be close to each other? Do the overlap of the circles mean anything or are they a consequence of the force-directed algorithm? 

2) In Table 1, you show the outputs of your method for the description "The molecule is a primary aliphatic ammonium ion which is obtained from streptothricin F by protonation of the guanidino and amino groups. It has a role as an antimicrobial agent. It is a guanidinium ion and a primary aliphatic ammonium ion. It is a conjugate acid of a streptothricin F". Are all the generated molecules a conjugate acid of  streptothricin F? Do they have the correct scaffold? This would be important to understand and is related to limitations of the BLEU score.
**Originality**
To my knowledge, the method is novel and the problem is under-researched.

**Quality**
The SFT and RL methods are clear and intuitive. The algorithms are well described in the text and code is provided in Supplementary. Figure 4 helps the understanding of the method. The gap in existing LLMs at generating structurally diverse molecules is well explained in the text and in figure 2. The baselines and metrics are mostly well chosen. The results are convincing, with the authors' method producing significant increase in the chosen metrics.

**Clarity** The methods are explained well. The figures are easy to read. Including different texture in Figure 5 helps the readability. The Appendix with all the experimental details and the code in Supplementary is appreciated and helps reproducibility. I also appreciate the title and claims being straightforward and to the point. 

**Significance** Generating diverse molecules is important in discovery workflows, thus the problem here is significant. The proposed method is easy to understand and implement. The paper also inspires a few interesting follow-ups and thus I think is an important addition to the community.
The paper proposes a fine-tuning method to generate diverse molecules using LLMs. The method involves supervised fine-tuning (SFT) and reinforcement learning (RL).

For SFT, the LLM is prompted to generate a molecule with a given property. The prompt is sampled many times, the generated molecules are filtered and then concatenated to create the SFT training set. The SFT step uses a prompt describing the property and requesting a set of molecules. The set of molecules generated previously is appended to this prompt.

For RL, the SFT prompt is used to generate a sequence of molecules. Every time a molecule is generated, the LLM policy is updated using Proximal Policy Optimization (PPO). The reward is based on how well the molecule matches a property and how different it is from the previous molecules.
1) In the experiments with BioT5+ and MolT5, the authors used the BLEU score on prompts and molecules from ChEBI-20. I think BLEU is a misleading score in the text to molecule task. For example, in BLEU, the order of words in a sentence matters, while there are many ways of writing a molecules as SMILES strings. Also, small changes in a molecule SMILES in the right place can have high changes in a property, such as in the case of hydrogen bond donors and acceptors. I recommend the authors discuss this in their manuscript and compare it with scores generated by RDKit (which they used in the other experiments). 

2) There are a few minor things that could improve the paper: 
    - Figure 7 could have different symbols. It was hard to read in a black-and-white version of the paper.   
    - Worth considering giving the method a name, this will help wider adoption.
Yes
No ethics review needed.
1. **Cluster Representation in Figure 6**:
   Is it reasonable to represent each cluster as a circle in Figure 6? Given that a 2D projection of a circle in a high-dimensional space may not accurately be circular, I wonder if this visualization accurately reflects clustering in chemical space.
2. **Effect of SMILES Randomness on BLEU Scores**:
   Since a single molecule can be represented by multiple SMILES strings, the randomness in SMILES representations could impact BLEU scores in the experiments. Has this potential source of variability been considered, and what effect does it have on the method’s stability or the evaluation of generated molecules?

In conclusion, this paper offers a valuable approach to improving molecular diversity in AI-driven drug discovery, and I will raise my score if my concerns (in Weaknesses and Questions) are well-addressed.
1. **Focus on an Essential Problem**: The paper addresses a critical requirement in drug discovery—molecular diversity in generated candidates. By providing a more diverse set of molecules, the proposed method aligns with real-world drug discovery processes, enhancing the chance of identifying successful compounds.
2. **Clear Presentation**: The paper is well-structured, with a clear exposition of the methodology and experimental procedures.
3. **Detailed Experiments**: The authors perform thorough evaluations across multiple metrics and baselines, including the comparison of Div-SFT+RL with various decoding schemes. The use of IntDiv, and NCircles as diversity-related metrics offers a comprehensive view of the method’s performance.
This paper tackles an essential problem in AI-driven drug discovery: generating structurally diverse molecules using LLMs. Addressing the challenge that traditional LLMs often produce structurally similar molecules from a single prompt, the authors propose a two-stage approach: (1) supervised fine-tuning to enable autoregressive mutiple molecule generation, and (2) RL to enhance structural diversity. Through experiments with several LLMs and comparison against various decoding schemes, the proposed approach, Div-SFT+RL, demonstrates an improved ability to generate diverse molecular structures, which is crucial for discovering viable drug candidates. The results indicate that Div-SFT+RL outperforms existing models on diversity-related metrics, supporting its potential for applications in drug discovery.
1. **Limited Discussion of Related Works on RL for Diversity in Molecular Generation**:
   The paper lacks an in-depth discussion of prior reinforcement learning (RL) approaches that also aim to improve molecular diversity, particularly those using RL for targeted or diverse molecule generation. Works such as Blaschke et al. (2020), Pereira et al. (2021), Hu et al. (2023), and He et al. (2024) provide diverse RL strategies in molecular design, which may not all involve LLMs but offer relevant insights on RL methods and should be discussed to position this work better.

   [1] Blaschke, Thomas, et al. "Memory-assisted reinforcement learning for diverse molecular de novo design." Journal of cheminformatics 12.1 (2020): 68. 

   [2] Pereira, Tiago, et al. "Diversity oriented deep reinforcement learning for targeted molecule generation." Journal of cheminformatics 13.1 (2021): 21. 

   [3] Hu, Xiuyuan, et al. "De novo drug design using reinforcement learning with multiple gpt agents." Advances in Neural Information Processing Systems 36 (2023). 

   [4] He, Jiazhen, et al. "Evaluation of reinforcement learning in transformer-based molecular design." Journal of Cheminformatics 16.1 (2024): 95.

2. **Applicability of the ChEBI-20 Dataset for Diversity Evaluation**:
   The use of the ChEBI-20 dataset raises concerns, as each molecular description in this dataset corresponds to a single molecule. Baseline approaches typically focus on generating one molecule to match the target rather than generating a diverse set for a single description. Furthermore, some descriptions may specify molecular details to the extent that producing diverse structures could be contradictory, making ChEBI-20 potentially unsuitable for assessing molecular diversity.

3. **Reward Function’s Balancing of Terms**:
   The reward function includes two terms: one for structural diversity and another for description matching. It would be beneficial to introduce a coefficient to balance these terms, enabling tuning based on the importance of each aspect (diversity vs. adherence to the description).

4. **NCircles Values in Figure 5**:
   In Figure 5, NCircles values at threshold $h=0.75$ are higher than those at $h=0.65$. This seems inconsistent with the metric’s definition, where values at a higher threshold should logically be lower. The unexpected result suggests potential misinterpretation or misuse of the metric, which requires clarification.
We propose a fine-tuning approach for LLMs to autoregressively generate a diverse set of molecules.
@inproceedings{
anonymous2024can,
title={Can {LLM}s Generate Diverse Molecules? Towards Alignment with Structural Diversity},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=B9177IHxCL},
note={under review}
}
Recent advancements in large language models (LLMs) have demonstrated impressive performance in generating molecular structures as drug candidates, which offers significant potential to accelerate drug discovery. However, the current LLMs overlook a critical requirement for drug discovery: proposing a diverse set of molecules. This diversity is essential for improving the chances of finding a viable drug, as it provides alternative molecules that may succeed where others fail in wet-lab or clinical validations. Despite such a need for diversity, the LLMs often output structurally similar molecules from a given prompt. While decoding schemes like beam search may enhance textual diversity, this often does not align with molecular structural diversity. In response, we propose a new method for fine-tuning molecular generative LLMs to autoregressively generate a set of structurally diverse molecules, where each molecule is generated by conditioning on the previously generated molecules. Our approach consists of two stages: (1) supervised fine-tuning to adapt LLMs to autoregressively generate molecules in a sequence and (2) reinforcement learning to maximize structural diversity within the generated molecules. Our experiments show that (1) our fine-tuning approach enables the LLMs to better discover diverse molecules compared to existing decoding schemes and (2) our fine-tuned model outperforms other representative LLMs in generating diverse molecules, including the ones fine-tuned on chemical domains.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large language model molecular generative model drug discovery
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/246d8eec44726b22790bbe651bcb52b452f42365.pdf
applications to computer vision, audio, language, and other modalities
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/5121e392d054edf309ef3365c9fcbb425d94d31c.zip
Can LLMs Generate Diverse Molecules? Towards Alignment with Structural Diversity
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission