Title: Measuring Software Performance on Linux,
Abstract: Measuring and analyzing the performance of software has reached a high
complexity, caused by more advanced processor designs and the intricate
interaction between user programs, the operating system, and the processor's
microarchitecture. In this report, we summarize our experience about how
performance characteristics of software should be measured when running on a
Linux operating system and a modern processor. In particular, (1) We provide a
general overview about hardware and operating system features that may have a
significant impact on timing and how they interact, (2) we identify sources of
errors that need to be controlled in order to obtain unbiased measurement
results, and (3) we propose a measurement setup for Linux to minimize errors.
Although not the focus of this report, we describe the measurement process
using hardware performance counters, which can faithfully reflect the real
bottlenecks on a given processor. Our experiments confirm that our measurement
setup has a large impact on the results. More surprisingly, however, they also
suggest that the setup can be negligible for certain analysis methods.
Furthermore, we found that our setup maintains significantly better performance
under background load conditions, which means it can be used to improve
software in high-performance applications.