What does a joint probability density function have to do with Stochastic Optimal Control and Reinforcement Learning?
Extracting a joint PDF just means that you create a model that models the behavior of several variables combined instead of in isolation. 

If these variables aren't independent and your loss functions is influenced by all of them, you obviously have to learn this joint PDF to minimize your loss. 

So I don't see this statement as particularly mysterious.