reference request: solving problems by dynamic programming + quantization to avoid combinatorial explosion
This sounds like a reinforcement learning problem.
Construct the following MDP:

The states are the different values of $x$
The actions are $u_1,\dots,u_L$
The transition is given by $f$
The reward is given by $g$ of the current state.

Define it as a finite horizon problem.
Then, if the set of the values of $x$ is finite, use any RL technique to solve the problem (you can use a dynamic programming solution from there to compute the exact solution).
If the state space is infinite, or extremely huge, consider trying to use the equivalent techniques from reinforcement learning.
If you wish, leave here a comment and when I will be able to I will send the algorithms themselves.

Anyways, from my understanding of your solution - it looks correct. But you need to be cautious about the use of $B$ and $\epsilon$ since cutting the space into boxes increases the number of boxes dramatically with the increase of the dimensions