However, this reasoning has been questioned. Giving consistent results is not always equal to giving the correct results. In general, it is more rigorous to assign weights to each of the programs, calculate the average weighted execution time (using the arithmetic mean), and then normalize that result to one of the computers. The three tables above just give a different weight to each of the programs, explaining the inconsistent results of the arithmetic and harmonic means (Table 4 gives equal weight to both programs, the Table 2 gives a weight of 1/1000 to the second program, and the Table 3 gives a weight of 1/100 to the second program and 1/10 to the first one). The use of the geometric mean for aggregating performance numbers should be avoided if possible, because multiplying execution times has no physical meaning, in contrast to adding times as in the arithmetic mean. Metrics that are inversely proportional to time (speedup, IPC) should be averaged using the harmonic mean.
However this reasoning has been questioned Giving consistent results is not always equal to giving the correct results In general it is more rigorous to assign weights to each of the programs calculate the average weighted execution time (using the arithmetic mean) and then normalize that result to one of the computers The three tables above just give a different weight to each of the programs explaining the inconsistent results of the arithmetic and harmonic means (Table 4 gives equal weight to both programs the Table 2 gives a weight of 1/1000 to the second program and the Table 3 gives a weight of 1/100 to the second program and 1/10 to the first one) The use of the geometric mean for aggregating performance numbers should be avoided if possible because multiplying execution times has no physical meaning in contrast to adding times as in the arithmetic mean Metrics that are inversely proportional to time (speedup IPC) should be averaged using the harmonic mean