How to compute the target for double Q-learning update step?
$$Y_{t}^{\text {DoubleDQN }} \equiv R_{t+1}+\gamma Q\left(S_{t+1}, \underset{a}{\operatorname{argmax}} Q\left(S_{t+1}, a ; \boldsymbol{\theta}_{t}\right), \boldsymbol{\theta}_{t}^{-}\right)$$
The only difference between the &quot;original&quot; DQN and this one is that you use your $Q_\text{est}$ with the next state to get your action (by choosing the action with the highest Q).
Afterward, you just figure out what the target $Q$ is given that action, by selecting the $Q$ belonging to that action from the target_network (instead of using the argmax a directly on the target Q network).
About the formula

$\theta_{t}^{-}$ above it means frozen weights, so it represents the target Q network.

the other $\theta_{t}$ represents the &quot;learnable weights&quot; so the estimate Q network.