Why is the reward in reinforcement learning always a scalar?
Rather than the survey by Liu et al. recommended above, I'd suggest you read the following survey paper for an overview of MORL (disclaimer - I was a co-author on this, but I genuinely think it is a much more useful introduction to this area)

Roijers, D. M., Vamplew, P., Whiteson, S., &amp; Dazeley, R. (2013). A survey of multi-objective sequential decision-making. Journal of Artificial Intelligence Research, 48, 67-113.

Liu et al's survey, in my opinion, doesn't do much more than list and briefly describe the MORL algorithms which existed at that point. There's no deeper analysis of the field. The original version of their paper was also retracted due to blatant plagiarism of several other authors, including myself as can be confirmed here: https://ieeexplore.ieee.org/document/6509978
Our survey provides arguments for the need for multiobjective methods by describing 3 scenarios where agents using single-objective RL may be unable to provide a satisfactory solution that matches the needs of the user. Briefly, these are

the unknown weights scenario where the required trade-off between the objectives isn't known in advance, and so to be effective the agent must learn multiple policies corresponding to different trade-offs and then at run-time select the one which matches the current preferences (e.g. this can arise when the objectives correspond to different costs which vary in relative price over time);

the decision support scenario where scalarization of a reward vector is not viable (for example, in the case of subjective preferences, which defy explicit quantification), so the agent needs to learn a set of policies, and then present these to a user who will select their preferred option, and

the known weights scenario where the desired trade-off between objectives is known, but its nature is such that the returns are non-additive (i.e. if the user's utility function is non-linear), and therefore standard single-objective methods based on the Bellman equation can't be directly applied.


We propose a taxonomy of MORL problems in terms of the number of policies they require (single or multi-policy), the form of utility/scalarization function supported (linear or non-linear), and whether deterministic or stochastic policies are allowed, and relate this to the nature of the set of solutions which the MO algorithm needs to output. This taxonomy is then used to categorise existing MO planning and MORL methods.
One final important contribution is identifying the distinction between maximising Expected Scalarised Return (ESR) or Scalarised Expected Return (SER). The former is appropriate in cases where we are concerned about the results within each individual episode (for example, when treating a patient - that patient will only care about their own individual experience), while SER is appropriate if we care about the average return over multiple episodes. This has turned out to be a much more important issue than I anticipated at the time of the survey, and Diederik Roijers and his colleagues have examined it more closely since then (e.g. http://roijers.info/pub/esr_paper.pdf)