Why do game theorists use a discounted payoff of this form?
In my experience, it's mainly just for cleanliness for results. 

Consider an infinite horizon repeated game, with discounted payoff representation (where I use $\delta = (1-\lambda)$ in your notation)
$$
(1-\delta)\sum_{t=0}^{\infty}\delta^t R_t
$$
where $0 &lt; \delta &lt; 1$. 

Suppose I play a strategy that gives me the same payoff, say $a$, for each period $t$. Then,
$$
(1-\delta)\sum_{t=0}^{\infty}\delta^t a = (1-\delta)\frac{a}{1-\delta} = a
$$
which is cleaner than
$$
\frac{a}{1-\delta}
$$
As a side note, multiplying the utility by a constant does not change preferences, so we keep the inherent preferences the same.