An extremely well-studied formulation in stochastic control is that of linear quadratic Gaussian control.
Any deviation from the above assumptions-a nonlinear state equation, a non-quadratic objective function, noise in the multiplicative parameters of the model, or decentralization of control-causes the certainty equivalence property not to hold.