Are Q values estimated from a DQN different from a duelling DQN with the same number of layers and filters?
Dueling-DQN has different network architecture comparing to vanilla DQN, so I don't think your version will work as well as the Dueling architecture.




  From Wang et al., 2016, Dueling Network Architectures for Deep Reinforcement Learning


On the other hand, since we only have the target Q-value, separating the Q-value into state value and advantage result in the identifiability issue. That is the network might simply learn $V(s)=0$, $A(s,a)=Q(s,a)$ for every state.

To tackle with this issue, we should force an additional constraint on the advantage estimate. We can simply use the equation below as mentioned in the paper, that is, normalize the advantages across actions before combining with the state value.

$$Q(s,a;\theta,\alpha,\beta)=V(s;\theta,\beta)+(A(s,a;\theta,\alpha)-\frac{1}{|A|}\sum\limits_{a'}A(s,a';\theta,\alpha))$$