Is there a way of path reconstruction using only the history of belief state?
The belief state in a POMDP is a distribution over the hidden state given all past actions and observations, i.e., at time $k$, the belief state is $b_k(s_k) \triangleq P(s_k \mid a_{0:k-1}, z_{1:k})$, where $a$ and $z$ are the actions and observations, respectively.
What you are asking about and calling &quot;backtracking&quot; boils down to the question: &quot;given that at time $k$, I know the history of actions $a_{0:k-1}$ and observations $z_{1:k}$, what is the distribution over states at some past time step $t&lt;k$, that is, $P(s_t\mid a_{0:k-1},z_{1:k})$&quot;?
This is commonly known as Bayesian smoothing. You might be familiar with Bayesian filtering, which aims to recover the belief state as defined in the first paragraph - an estimate of the current state given actions and observations up to the current time. Smoothing also uses information gained after time $t$ to estimate the state at time $t$, and hence it is only possible after those actions and observations become known. There is also prediction, where you estimate the distribution over the state in the future.
Smoothing is too broad of a topic to go into detail here, but below is a picture from the book &quot;Bayesian Filtering and Smoothing&quot; by Simo Särkkä that illustrates the concepts. You could start there for further information on the topic.