Benchmarking Safety Monitors for Image Classifiers with Machine Learning
  High-accurate machine learning (ML) image classifiers cannot guarantee that
they will not fail at operation. Thus, their deployment in safety-critical
applications such as autonomous vehicles is still an open issue. The use of
fault tolerance mechanisms such as safety monitors is a promising direction to
keep the system in a safe state despite errors of the ML classifier. As the
prediction from the ML is the core information directly impacting safety, many
works are focusing on monitoring the ML model itself. Checking the efficiency
of such monitors in the context of safety-critical applications is thus a
significant challenge. Therefore, this paper aims at establishing a baseline
framework for benchmarking monitors for ML image classifiers. Furthermore, we
propose a framework covering the entire pipeline, from data generation to
evaluation. Our approach measures monitor performance with a broader set of
metrics than usually proposed in the literature. Moreover, we benchmark three
different monitor approaches in 79 benchmark datasets containing five
categories of out-of-distribution data for image classifiers: class novelty,
noise, anomalies, distributional shifts, and adversarial attacks. Our results
indicate that these monitors are no more accurate than a random monitor. We
also release the code of all experiments for reproducibility.
