Adaptive information-theoretic bounded rational decision-making with
  parametric priors
  Deviations from rational decision-making due to limited computational
resources have been studied in the field of bounded rationality, originally
proposed by Herbert Simon. There have been a number of different approaches to
model bounded rationality ranging from optimality principles to heuristics.
Here we take an information-theoretic approach to bounded rationality, where
information-processing costs are measured by the relative entropy between a
posterior decision strategy and a given fixed prior strategy. In the case of
multiple environments, it can be shown that there is an optimal prior rendering
the bounded rationality problem equivalent to the rate distortion problem for
lossy compression in information theory. Accordingly, the optimal prior and
posterior strategies can be computed by the well-known Blahut-Arimoto algorithm
which requires the computation of partition sums over all possible outcomes and
cannot be applied straightforwardly to continuous problems. Here we derive a
sampling-based alternative update rule for the adaptation of prior behaviors of
decision-makers and we show convergence to the optimal prior predicted by rate
distortion theory. Importantly, the update rule avoids typical infeasible
operations such as the computation of partition sums. We show in simulations a
proof of concept for discrete action and environment domains. This approach is
not only interesting as a generic computational method, but might also provide
a more realistic model of human decision-making processes occurring on a fast
and a slow time scale.
