Additionally, we recognize that one interesting benefit of working with experimental data is that we know when a given structure was first reported; the data is time-stamped. Inspired by common practice in time-series forecasting, we hence can ask "could we predict the performance of materials discovered after year X if we only trained on materials discovered before year X?" In particular, we can measure how many of the top k materials we can recover in the top n predictions by the model. Using mofdscribe, this question can easily be answered. Fig. | Illustration of some splitting approaches implemented in mofdscribe. For the validation of a machine learning model, it needs to be tested on data it has not seen before. To create sets of such unseen data, datasets are typically split into subsets for training, testing (and validation). The conventional approach is to perform this split randomly. This, however, might not be a good approximation of real-world use with dependent/grouped data, distribution shift, or lead to problems in the case of imbalanced data. Therefore, mofdscribe implements various splitting strategies that either operate on (structural) features or other metadata. For instance, we extracted the publication date for all structures we could trace back to the CSD and hence allow performing a time-based split. Alternatively, one can use structural features to either ensure equal distribution of the features in different splits (stratification) or to mimic a distribution shift/extrapolation case by forcing different groups into different folds of a cross-validation scheme. The easiest example is to use the density; however, one can also cluster (we use k-means clustering after PCA) on features, e.g., computed using mofdscribe). Moreover, we also implement a splitting strategy inspired by the scaffold splits sometimes used for molecules for reticular molecules via our hash strategies (Figure ). This allows grouping structures with the same connectivity or backbone into the same fold. A different strategy is using Kennard-Stone sampling, to ensure that the training set is maximally diverse. Importantly, a time-based split is not the only feasible splitting strategy-and, depending on the use case, might not be the best option (or might not be applicable if no timestamps are available). Therefore, to further ensure that thorough model evaluation becomes routine for digital reticular chemistry, mofdscribe also implements, inspired by the DeepChem library, a variety Splitter classes (Figure , Listing 3 for a usage example). The Splitter classes either take a Listing 3 | Example of the use of Splitters. The datasets implemented in mofdscribe already provide the relevant information for the splitters (e.g. times, hashes, densities). If the splitters are used on other structures, e.g. custom in-silico assembled MOFs, this information will be computed, if possible on the fly, or can be provided by the user. Note that the datasets, by default, are deduplicated based on the graph hash. where one can see that the average generalization performance of models significantly depends on the splitting strategy. Overall, we find that the optimal split depends on the task at hand-but typically is not the conventional random split. Given the redundancy in scaffolds in MOF databases, we urge practitioners to use grouped cross-validation.