Why is it that the state visitation frequency equals the sum of state visitation frequency from initial time step to the horizon? In the maximum entropy inverse reinforcement learning paper, Ziebart et al. show that the state visitation frequency $\rho(s)$ of a state $s$ can be computed as
$$
\rho_{\pi}(s) = \sum_{t}^{T} P(s_t=s|\pi),
$$
which is the sum of the probability that the state being visited at each time step.
I just don't understand why is it the sum? From my perspective, a frequency should be the less than one, so that it should be the average value
$$
\rho_{\pi}(s) = \frac{1}{T}\sum_{t}^{T} P(s_t=s|\pi).
$$
It should be the average and this is rarely mentioned by people except for a IRL summer camp at UCB. You can check this GithubIssue for details.