 From the theory of MDPs it is known that, without loss of generality, the search can be restricted to the set of so-called "stationary" policies.A policy is "stationary" if the action-distribution returned by it depends only on the last state visited (from the observation agent's history).The search can be further restricted to "deterministic" stationary policies.