Is there a way to solve an optimization problem where a decision variable shows up in an upper bound (or lower bound) of summation?
This problem is strongly reminiscent of "Optimal stopping" of stochastic processes.  These are frequently solved by dynamic programming.  References even to this specialized literature are extensive, due to its applications in decision theory and finance.