Solution Methods for Constrained Markov Decision Process with Continuous
  Probability Modulation
  We propose solution methods for previously-unsolved constrained MDPs in which
actions can continuously modify the transition probabilities within some
acceptable sets. While many methods have been proposed to solve regular MDPs
with large state sets, there are few practical approaches for solving
constrained MDPs with large action sets. In particular, we show that the
continuous action sets can be replaced by their extreme points when the rewards
are linear in the modulation. We also develop a tractable optimization
formulation for concave reward functions and, surprisingly, also extend it to
non- concave reward functions by using their concave envelopes. We evaluate the
effectiveness of the approach on the problem of managing delinquencies in a
portfolio of loans.
