Why is it that the state visitation frequency equals the sum of state visitation frequency from initial time step to the horizon?
It should be the average and this is rarely mentioned by people except for a IRL summer camp at UCB. You can check this GithubIssue for details.