How to prove the formula of eligibility traces operator in reinforcement learning?
I will refer to $\mathcal T^{\pi} $as $\mathcal T$ and $P^{\pi}$ as $P$ for notational simplicity
\begin{align}
(\mathcal{T})^{n+1} Q &amp;= \mathcal{T}(\mathcal{T}(...(\mathcal{T}(Q))))\\
&amp;= r + \gamma P(r + \gamma P(...(r + \gamma P Q)))\\
&amp;= r + r\sum_{i=1}^{n} \gamma^i P^i + \gamma^{n+1} P^{n+1} Q
\end{align}
\begin{align}
\mathcal{T}_{\lambda}Q &amp;= (1-\lambda) \sum_{n=0}^{\infty} \lambda^n (\mathcal{T})^{n+1} Q\\
&amp;=(1-\lambda)\{\lambda^0 (\mathcal T)^1Q + \lambda^1 (\mathcal T)^2Q + \lambda^2 (\mathcal T)^3Q + \ldots   \}
\end{align}
when you plug in expression for $(\mathcal T)^i Q$ inside this sum and rearrange you get 3 sums
\begin{equation}
\mathcal{T}_{\lambda}Q = (1-\lambda) \sum_{n=0}^{\infty} \lambda^n r + (1-\lambda)\sum_{n=1}^{\infty} \lambda^n \gamma^n P^n r + (1-\lambda)\sum_{n=0}^{\infty} \lambda^n \gamma^{n+1} P^{n+1} Q
\end{equation}

sum:
\begin{equation}
(1-\lambda) \sum_{n=0}^{\infty} \lambda^n r = r
\end{equation}
sum:
\begin{equation}
(1-\lambda)\sum_{n=1}^{k} \lambda^n \gamma^n P^n r = (1-\lambda\gamma P)^{-1}(1 - \lambda^k \gamma^k P^k)\lambda\gamma P r
\end{equation}
As $k \rightarrow \infty$ and since $\gamma &lt; 1$ this is in the limit equal to
\begin{equation}
(1-\lambda)\sum_{n=1}^{\infty} \lambda^n \gamma^n P^n r = (1-\lambda\gamma P)^{-1}\lambda\gamma P r
\end{equation}
sum:
\begin{equation}
(1-\lambda)\sum_{n=0}^{\infty} \lambda^n \gamma^{n+1} P^{n+1} Q = (1 - \gamma\lambda P)^{-1}(1-\lambda)\gamma P Q 
\end{equation}
If you combine all 3 you get
\begin{align}
\mathcal{T}_{\lambda}Q &amp;= r + (1-\lambda\gamma P)^{-1}\lambda\gamma P r + (1 - \gamma\lambda P)^{-1}(1-\lambda)\gamma P Q\\
&amp;= r+ (1-\lambda\gamma P)^{-1}(\lambda \gamma P r + \gamma PQ - \lambda\gamma PQ)\\
&amp;= r+ (1-\lambda\gamma P)^{-1}(\lambda \gamma P r + (\mathcal T)Q - r - \lambda\gamma PQ)\\
&amp;= (1-\lambda\gamma P)^{-1}(r - \lambda \gamma P r + \lambda \gamma P r + (\mathcal T)Q - r - \lambda\gamma PQ)\\
&amp;= (1-\lambda\gamma P)^{-1}( (\mathcal T)Q  - \lambda\gamma PQ + Q - Q)\\
&amp;= Q + (1-\lambda\gamma P)^{-1}( (\mathcal T)Q  - Q)
\end{align}