Why is the target called "target" in Monte Carlo and TD learning if it is not the true target?
It is our &quot;current&quot; target. We assume that the value we get now is at least a closer approximation to the &quot;true&quot; target.
We're not so much moving towards a wrong value as we are moving away from a more wrong value.
Of course, it is all base on random trials, so saying anything definite (such as: &quot;we are guaranteed to improve at each step&quot;) is hard to show without working probabilistically. The expectation of the error of the value function (as compared to the true value function) will decrease, that is all we can say.