We have shown that the epistemic uncertainty of a single prediction obtained via a committee standard deviation cannot be directly correlated to the absolute error, reiterating the well-known asymmetric relation between error and uncertainty. For machine learning potentials, this holds true both for energy and atomic force predictions. Building on these results, we have developed an approach to nevertheless use force uncertainties to identify high-error data points by aggregating force uncertainties and errors over groups of atoms. For variancedominated models, a strong correlation between aggregated uncertainties and errors can be proven mathematically. For machine learning potentials, we further find that the approach holds for models containing a substantial amount of bias. For such models, we observe that the presence of bias amplifies the proportionality factor between aggregated uncertainties and errors compared to the variance-only case. Since the proportionality factor needs to be obtained from a fit to a validation set for any real-world scenario anyways, this does not impede the reliability of the proposed method. The aggregated force uncertainty is directly correlated both with the aggregated absolute error over that group of atoms but also the total absolute error of the model prediction for various datasets. Our approach confidently identifies high-error data points for systems with low and high numbers of atoms, and is applicable to periodic and nonperiodic systems in gas, liquid, and solid phase alike. We furthermore demonstrated that an aggregation must not necessarily include all atoms of a structure, but can be restricted to neighboring atoms around an atom of interest. Locally aggregated uncertainties can then be applied to identify high-error local substructures, and thus resolve absolute errors on an atomic scale. The benefits of locally aggregated uncertainties were showcased for an active learning study, indicating that data selection via spatially resolved uncertainties allows for a data-efficient and fast training of accurate machine learning potentials. We therefore envision this workflow to be very powerful for active learning frameworks in low-data regimes for large, demanding systems, such as multi-phase systems.