In reinforcement learning, it is essential to consider the trade-off between exploration and exploitation to achieve optimal performance. Exploitation involves selecting actions that maximize immediate rewards and generate MOFs similar to those created by the agent in the pre-training stage. On the other hand, exploration involves selecting actions that explore the action space for long-term benefits, generating novel MOF structures that have not been previously seen. Achieving this balance is critical to the success of the reinforcement learning algorithm. To address this challenge, we introduced two generators: one for exploration, and one biased towards exploitation (see Fig. ). The generator was updated to maximize rewards through the reinforcement learning algorithm, while the weights of the biased generator were frozen to enable continuous generation of MOFs by the pre-training generator. During training, each token of the MOF representations was selected from either the biased generator or the generator, with the choice determined by a threshold λ. A higher value of λ favored exploitation over exploration, and an empirical observation determined λ to be set at 0.5. Moreover, to improve the diversity of topology and metal clusters of the generated MOF structures, 50 % of the metal clusters in the inputs were masked during the training stage of reinforcement learning, allowing the decoder of the generator to select topologies and metal clusters without considering the metal clusters used as inputs.