is faced with a stochastic optimization problem. Let the interest r follow a Markov process with probability transition function where denotes the probability measure governing the distribution of interest rate next period if current interest rate is . In this model the consumer decides their current period consumption after the current period interest rate is announced. Rather than simply choosing a single sequence , the consumer now must choose a sequence for each possible realization of a in such a way that their lifetime expected utility is maximized: The expectation is taken with respect to the appropriate probability measure given by Q on the sequences of rs. Because r is governed by a Markov process, dynamic programming simplifies the problem significantly. Then the Bellman equation is simply: Under some reasonable assumption, the resulting optimal policy function g(a,r) is measurable. For a general stochastic sequential optimization problem with Markovian shocks and where the agent is faced with their decision ex-post, the Bellman equation takes a very similar form Solution methods The method of undetermined coefficients, also known as 'guess and verify', can be used to solve some infinite-horizon, autonomous Bellman equations. The Bellman equation can be solved by backwards induction, either analytically in a few special cases, or numerically on a computer. Numerical backwards induction is applicable to a wide variety of problems, but may be infeasible when there are many state variables, due to the curse of dimensionality. Approximate dynamic programming has been introduced by D. P. Bertsekas and J. N. Tsitsiklis with the use of artificial neural networks (multilayer perceptrons) for approximating the Bellman function. This is an effective mitigation strategy for reducing the impact of dimensionality by replacing the memorization of the complete function mapping for the whole space domain with the memorization of the sole neural network parameters. In particular, for continuous-time systems, an approximate dynamic programming approach that combines both policy iterations with neural networks was introduced. In discrete-time, an approach to solve the HJB equation combining value iterations and neural networks was introduced. By calculating the first-order conditions associated with the Bellman equation, and then using the envelope theorem to eliminate the derivatives of the value function, it is possible to obtain a system of difference equations or differential equations called the 'Euler equations'. Standard techniques for the solution of difference or differential equations can then be used to calculate the dynamics of the state