System self-tuning is a crucial task to lower the energy consumption of computers. Traditional approaches decrease the processor frequency in idle or synchronisation periods. However, in High-Performance Computing (HPC) this is not sufficient: if the executed code is load balanced, there are neither idle nor synchronisation phases that can be exploited. Therefore, alternative self-tuning approaches are needed, which allow exploiting different compute characteristics of HPC programs. The novel notion of application regions based on function call stacks, introduced in the Horizon 2020 Project READEX, allows us to define such a self-tuning approach. In this paper, we combine these regions with the Q-Learning typical state-action maps, which save information about available states, possible actions to take, and the expected rewards. By exploiting the existing processor power interface, we are able to provide direct feedback to the learning process. This approach allows us to save up to 15 energy, while only adding a minor runtime overhead.
@cite_7: Dynamic voltage and frequency scaling (DVFS) has been widely used for energy reduction in the modern processors. How to select the optimal frequency that minimizes energy dissipation for the given performance constraint at runtime is a nontrivial problem. The problem becomes more complicated if temperature needs to be constrained (or minimized) simultaneously. The temperature, performance and energy have different nonlinear relationships with frequency voltage scaling ratio and this relationship is closely related to the characteristics of hardware and applications. In this paper, we design a reinforcement learning algorithm to tackle the problem of simultaneous temperature, performance and energy management. The proposed approach allows continuous tradeoff among these three quality measurement of a computer system. It also enables us to set two of the measurements as constraints and optimize the third one. The proposed approach is validated on an Intel Core 2 processor running Linux system.
Another example is given by Shen et. ,al. @cite_7 , who use Q-Learning to find a trade-off between temperature, performance, and energy. They define a state as a combination of instructions per second, stall cycles, the current processor frequency, and the current processor temperature. This state is captured and evaluated every 10 ,ms. Based on the observations, they calculate a reward, which is used to update the Q-value. Based on the information, an action is taken, i.e., the frequency is changed. Shen et. ,al. use models to evaluate the performance and energy impact of their decisions. However, these models can result in wrong predictions and have to be carefully chosen. By using well-defined program regions, we can measure the energy and performance impacts, which is more precise than a model evaluated every 10 ,ms.