How to choose between discounted reward and average reward?
The Average reward in that figure is used as a measure of performance. In other words, the score of the agent playing the game. You do not track reward per episode as this doesnt indicate a general improvement in the learning process. Instead you track the average reward over training epochs. If it steadily increases this means that your agent indeed is learning.

The discounted reward is used to create some kind of future-reward dependencies and is used in the learning equations. So, instead of evaluating how good is a particular state according to the immediate reward you received, you also take into account the future reward from your next state. In RL you attempt to max your expected return and some methods estimate the expected reward from every state.

Please note that my answer gives a high level description and is not referring to a specific RL algorithm (as there are many variations). I would suggest you to understand very well the simple tabular form of Q-learning before moving to RL and function approximators combinations.