Unable to understand V* at infinite time horizon using Bellman equation for solving MDP
Nevermind. I found that above answer is indeed correct, but the gradescope has a bug (it requires the format to be .2 instead of 0.2).