Two DQNs in two different time scales
From comments, you say there is no &quot;outer&quot; goal for picking an adversary other than  scoring highly in an individual episode.
You could potentially model the initial adversary choice as a partially separate Markov Decision Process (MDP), where choosing the opponent is a single-step episode with return equal to whatever reward the secondary MDP - which played the game - obtains. However, this &quot;outer&quot; MDP is not much of an MDP at all, it is more like a contextual bandit. In addition, the performance of the inner game-playing agent will vary both with the choice of opponent, and over time as it learns to play better against each opponent. This makes the outer MDP non-stationary. It also requires the inner MDP to know what opponent it is facing in order to correctly predict correct choices and/or future rewards.
That last part - the need for any &quot;inner&quot; agent to be aware of the opponent it is playing against - is likely to be necessary whatever structure you chooose. That choice of opponent needs to be part of the state for this inner agent, because it will have an impact on likely future rewards. A characterisation of the opponents also needs to be part of whatever predictive models you could use for the outer agent.
A more natural, and probably more useful, MDP model for your problem is to have a single MDP where the first action $a_0$ is to select the opponent. This matches the language you use to describe the problem, and resolves your issue about trying to run a hierarchy of agents. Hierarchical reinforcement learning is a real thing, and very interesting for solving problems which can be broken down into meaningful sub-goals that an agent could discover autonomously, but it does not appear to apply for your problem.
This leaves you with a practical problem of creating a model that can switch between choosing between two sets of radically different actions. The select an opponent action only occurs at the first state of the game, and the two sets of actions do not overlap at all. However, in terms of the theoretical MDP model this is not an issue at all. It is only a practical issue of how you get to fit your Q function approximator to the two radically different action types. There a a few ways around that. Here are a couple that might work for you:
One shared network
Always predict for all kinds of action choice, so the agent still makes predictions for switching opponents all the way to the end of the game. Then filter the action choices down to only those available at any time step. When $t=0$ only use the predictions for actions for selecting an opponent, for $t \ge 1$ only use predictions relating to moves in the game.
Two separate approximators
Have two function approximators in your agent, use one for predicting reward at $t=0$ that covers different opponent choices, and use the other for the rest of the game. If $n$ is small and there is no generalisation between opponents (i.e. no opponent &quot;stats&quot; that give some kind of clue towards the end results), then for the first approximator, you could even use a Q table.
For update steps you need to know whether any particular action value was modelled in one or other of the Q functions - and this will naturally lead you to bootstrap
$$\hat{q}_{o}(s_0, a_0, \theta_0) \leftarrow r_1 + \gamma \text{max}_{a'}\hat{q}_{p}(s_1, a', \theta_1)$$
where $\hat{q}_{o}$ is your approximate model for action values of selecting opponents (and $a_0$ must be an opponent choice) at the start of the game, and $\hat{q}_{p}$ is the nodel you use for the rest of it (and $a'$ must be a position play in the game). I've misused $\leftarrow$ here to stand in for whatever process used to update the action value towards the new estimate - a tabular method that would be a rolling average with current estimate, in neural networks of course that is gradient descent using backpropagation.