 Both algorithms compute a sequence of functions formula_63 (formula_64) that converge to formula_62.Computing these functions involves computing expectations over the whole state-space, which is impractical for all but the smallest (finite) MDPs.In reinforcement learning methods, expectations are approximated by averaging over samples and using function approximation techniques to cope with the need to represent value functions over large state-action spaces.