How to stop DQN Q function from increasing during learning?
You can use discount factor gamma less then one.
You can use finite time horizon - only for states which are no farther away then T time steps reward propagate back
You can use sum of rewords averaged over time for Q


All of those are legitimate approaches.