Limitations in the model architecture and representation can be easily overlooked if the dataset only spans a small subset of chemical space. This may be the case for databases including only molecules with the same number of atoms, or related chemical structures. As an example, we illustrate how a wrong choice in the aggregation function (which combines atomic into molecular embed- dings) for the QM9 target enthalpy can be overlooked if the size of the molecules in the dataset only spans a narrow range. To this aim, we split the QM9 dataset into molecules with 1-6 atoms and 7-9 atoms. The machine learning models are then trained solely using the dataset with 7-9 atoms. Fig. depicts the performance on test sets containing molecules of size 1-6 and 7-9 for a mean (top left) and sum (top right) aggregation function. For the test set containing similarly sized molecules, both aggregation functions lead to acceptable performances, and one might wrongly conclude that a mean aggregation function is a valid choice for an extensive target like the enthalpy. However, inspecting the test set performance on molecules of size 1-6 reveals that by using a mean aggregation function, the model does not gain any additional performance as more data is added. The bottom panels of Fig. depict the absolute errors for each datapoint in the test set as a function of molecular size. Here, the failure of the model utilizing mean aggregation becomes apparent: molecules with a different size produce absolute errors up to four orders of magnitude larger than molecules with a similar size because the model implicitly learns the average size of the molecules in the training set to circumvent the shortcoming of mean aggregation. We note that for sum aggregation, the extrapolation performance to differently sized molecules is by no means perfect (bottom right panel), but the model at least learns to generalize to some extent. In general, an extrapolation error is visible regardless of the model architecture. This extrapolation error can be assigned to bias by lack of data coverage for molecules with a lower molecular weight.