The results of this study corroborate the potential of multinomial sampling, not only for chemical space exploration to obtain chemically diverse molecular designs, but also for generating high-scoring compounds that are sufficiently diverse from the fine-tuning compounds. Multinomial sampling allows a user-defined number of SMILES strings (potentially exceeding 1000 molecules) to be generated via a CLM. It is reasonable to assume that better scoring molecules (i.e., with lower perplexity) can be created by generating more molecular designs via multinomial sampling. Assessing pretraining bias based on perplexity CLM pretraining might impose a greater effect on model performance than CLM fine-tuning, as model pretraining is typically performed with data that are at least two orders of magnitude higher in amount than fine-tuning . If a molecule is generated by a CLM due to pretraining only, it will not necessarily match the design objectives (as represented by the fine-tuning data). We analyzed the degree to which new molecules were generated due to the sole effect of pretraining, i.e., we verified whether "pretraining bias" occurred. In principle, for a CLM, perplexity can be used to score any molecule, including those that are not generated by the model. This can be achieved by computing the conditional probabilities of each SMILES character using the CLM. Therefore, the perplexity score was employed to differentiate between the information learned by the CLM during pretraining and during fine-tuning to score the molecules generated at a specified finetuning epoch. First, for each fine-tuning epoch, molecules were scored and ranked by the perplexity of the model used to generate them. Subsequently, each de novo design was scored and ranked based on the perplexity of the CLM after pretraining (i.e., prior to any fine-tuning). We hypothesized that a suitable ranking method should favor molecules that were generated based on information learned by the model during fine-tuning (capturing the final objectives of the experiment), and downrank the molecules generated based solely on pretraining (capturing "generic" information).