An important component of performance evaluation involves ensuring a proper separation between training and testing data. One simple way to accomplish this train/test split is to randomly split a large collection of annotated spectra, reserving some to train the model and some for testing. The nine-species benchmark dataset used in the original DeepNovo paper adopts this approach . This benchmark has been used extensively in subsequent studies . Unfortunately, this simple spectrumlevel splitting procedure does not ensure that peptide sequences seen during training do not also appear in the test set. Consequently, if a machine learning algorithm does a good job of "memorizing" the training set sequences, then using a spectrum-level split may give the algorithm an unfair advantage when presented with new spectra generated by the same peptides. To avoid this problem, some studies go a step further and carry out peptide-level splitting, thereby preventing leakage of sequence information from the training set to the test set . Because spectra originating from peptides that only differ by one or more PTMs are often highly similar to one another, this peptide-level splitting should be performed on the base peptide sequence irrespective of PTMs, i.e., a peptide and all its modified variants should be present in only a single split.