When should the last action be included in the state in reinforcement learning?
If your environment fulfills the Markov property, there is no reason to include the actions $a$ in the state $s$, as the action $a_t$ that lead to the new state $s_{t+1}$ should not provide any additional information, i.e. the knowledge of the old action should not influence the reward and transition functions from $s_{t+1}$ onwards.

Therefore, if your environment is a proper MDP, there is no reason to include actions in the next state.

It might be helpful in some rare cases, where your environment is not actually Markov to include the last action in your state, but in that case you should also think about if there is a better way to represent your state, such that it becomes Markov.

In your concrete case, where the actions do not change the state, there is no reason to include the last action as part of your state.

Side Note:
If your problem is a simple discrete bandit, you will probably be better off by looking into bandit approaches, rather than using a DQN based approach.