A neural network with 2 or more hidden layers is a DNN?
I don't think there is a fixed threshold that differentiates between Shallow and Deep Learning, but I would say that a 2 layer NN should not be considered deep. But now-a-days, almost all NN architectures are studied under the umbrella of Deep Learning.
And yes, training a 2 hidden layers NN using Q-learning would technically mean doing deep RL.
I guess it is conceptually simple but making NN perform optimally is an art. Tuning hyperparameters or debugging NN can be tough and one learns with experience. I guess others in the community would be much more suited to answer this question. But these were my 2 cents.