Depending on what you have as input, a policy or not, and what you want as output, optimal quantities or just the evaluation of a single policy, that determines what equations you use and what algorithm you use. 