* I posted this question [Cross Validated](https://stats.stackexchange.com/questions/555324/difference-between-regret-and-pseudo-regret-definitions), but didn't get any answer. So I am posting it here too, as the question is very relevant to machine learning

I am following the book [Bandit Algorithms](https://tor-lattimore.com/downloads/book/book.pdf). In page 48, they introduces regret after $n$ rounds as

$$
\mathbf{R} = n\mu^\star - \mathbb{E}\Bigg[\sum\_{t=1}^n \mathbf{X}\_t\Bigg] \tag{1}
$$

In page 55, they also define pseudo-regret as

$$
\bar{\mathbf{R}} = n\mu^\star - \sum\_{t=1}^n \mu\_{A\_t} \tag{2}
$$

In the paper [Regret Analysis of Stochastic and ...](https://arxiv.org/abs/1204.5721), authors introduces pseudo-regret as

$$
\bar{\mathbf{R}} = n\mu^\star - \mathbb{E}\Bigg[\sum\_{t=1}^n \mu\_{A\_t}\Bigg] \tag{3}
$$

Can anyone tell me the differences in the three definitions ?

In the first definition, due to the linearity of the expectation, we can write it exactly like (2). Hence (1) and (2) should be referring to the same quantity ?

Since $\mu\_{A\_t}$ is not a random variable, $\mathbb{E}[\mu\_{A\_t}] = \mu\_{A\_t}$, and due to the linearity of expectation, (2) and (3) should be referring to the same quantity.

Anyone can help me out with these definitions ?