Whence the Expected Free Energy?
  The Expected Free Energy (EFE) is a central quantity in the theory of active
inference. It is the quantity that all active inference agents are mandated to
minimize through action, and its decomposition into extrinsic and intrinsic
value terms is key to the balance of exploration and exploitation that active
inference agents evince. Despite its importance, the mathematical origins of
this quantity and its relation to the Variational Free Energy (VFE) remain
unclear. In this paper, we investigate the origins of the EFE in detail and
show that it is not simply "the free energy in the future". We present a
functional that we argue is the natural extension of the VFE, but which
actively discourages exploratory behaviour, thus demonstrating that exploration
does not directly follow from free energy minimization into the future. We then
develop a novel objective, the Free-Energy of the Expected Future (FEEF), which
possesses both the epistemic component of the EFE as well as an intuitive
mathematical grounding as the divergence between predicted and desired futures.
