Is deep learning appropriate to approximate dynamic programming problems?
A good start would be to understand what reinforcement learning is; and identify which problems are reinforcement learning problems and which aren't.  Deep reinforcement learning is only relevant if you have a reinforcement learning problem; otherwise, it's almost certainly not relevant.  Among other things, reinforcement learning deals with a stateful system.  You don't seem to have a stateful system so it's not clear to me why you think reinforcement learning would be relevant.

Since you haven't told us what the problem is, it's hard to imagine how we can tell you how to solve it; all I can share is general principles.  In general, deep reinforcement learning isn't magic that you use whenever a problem is too hard for normal methods; it is useful for a specific purpose, and I don't see any reason to expect it to be effective or applicable here.