contents tabular factors graphical models example belief networks inference methods recursive conditioning variable elimination stochastic simulation sampling from discrete distribution sampling methods for belief network inference rejection sampling likelihood weighting particle filtering examples gibbs sampling plotting behaviour of stochastic simulators hidden markov models exact filtering for hmms localization particle filtering for hmms generating examples dynamic belief networks representing dynamic belief networks unrolling dbns dbn filtering causal models planning with uncertainty decision networks example decision networks recursive conditioning for decision networks variable elimination for decision networks markov decision processes value iteration showing grid mdps asynchronous value iteration learning with uncertainty -means em reinforcement learning representing agents and environments simulating an environment from an mdp simple game evaluation and plotting version june