We consider the distribution of molecular properties of samples from top-k sampling and beam search with reference to an original training molecule. We find that the property distributions of samples vary considerably with sampling parameters, which introduce many degrees of freedom in the generation process . These include the number of beams, B, used in beam search, which explores a graph of generation possibilities by choosing samples that have the highest overall probability according to the model. Greedy search (not considered here) always selects the next token with the highest probability, hence beam search is guaranteed to find higher probability samples than a purely greedy strategy. The softmax temperature, T , controls the sensitivity of sampling to low probability tokens. Lowering T makes the distribution sharper, decreasing the likelihood of low probability tokens. For chemical generation, this results in less random, more repetitive samples. Conversely, increasing T results in more random, less repetitive samples. In top-k sampling , the k most likely next tokens are considered at each step of generation. This limits the sampling to a small set of high likelihood tokens, while expanding the diversity of sampling beyond greedy and beam search. In top-p (nucleus) sampling, the smallest set of tokens with cumulative probability > p are chosen. This dynamically changes the number of possible next tokens according to the changing probability distributions. We combine top-k and top-p sampling in our experiments, which avoids low probability tokens while providing more randomness and "surprising" outputs than beam search . We also vary the number of generated samples, n s , and the prompt length, l max (Figure ). To better understand the effects of sampling strategy, we repeat generation for ten different molecules randomly chosen from PubChem10M, and take prompts of length l max ∈ (5, 10, 20, l -3) tokens for each molecule, where l is the length of the original training molecule (Figure ). For the smallest and largest pre-trained ChemGPT models, we then plot the distributions of the percentages of samples that pass MOSES filters and the difference in molecular weight of generated samples compared to the original molecules used for conditioning (Figure ). The smaller model (green shading in Figure ) generally outperforms the larger model (orange) in generating samples that pass filters, for both top-k and beam search, except for k = 1000. However, the larger model generates samples that are more uniformly distributed with respect to molecular weight. The k parameter in top-k sampling has a limited effect for the large model, but tends to shift the samples to a lower pass filter rate for the small model. Changes in the beam size can induce shifts from multi-modal to unimodal distributions of samples. Overall, the property distributions of samples are highly dependent on the sampling strategy, sampling parameters, and model size, suggesting that the complexity of ChemGPT introduces many new and important degrees of freedom for generative modeling. Importantly, beam search is typically employed in sampling from chemical generative models and this restricts the diversity of outputs compared to top-k/nucleus sampling. Large chemical language models are self-supervised representation learners. A particularly exciting prospective application of large, pre-trained chemical models is to create a general representation learner that operates across diverse chemistries . To demonstrate the representation learning capabilities of ChemGPT, we show the illustrative examples of clustering and nearest-neighbor similarity search using unsupervised embeddings. We choose a representative dataset from Therapeutic Data Commons and MoleculeNet , the FreeSolv dataset of 642 druglike molecules and their hydration free energies in water.