What is the difference between Q-learning, Deep Q-learning and Deep Q-network?
Here is a table that attempts to systematically show the differences between tabular Q-learning (TQL), deep Q-learning (DQL), and deep Q-network (DQN).





Tabular Q-learning (TQL)
Deep Q-learning (DQL)
Deep Q-network (DQN)




Is it an RL algorithm?
Yes
Yes
No (unless you use DQN to refer to DQL, which is done often!)


Does it use neural networks?
No. It uses a table.
Yes
No. DQN is the neural network.


Is it a model?
No
No
Yes (but usually not in the RL sense)


Can it deal with continuous state spaces?
No (unless you discretize them)
Yes
Yes (in the sense that it can get real-valued inputs for the states)


Can it deal with continuous action spaces?
Yes (but maybe not a good idea)
Yes (but maybe not a good idea)
Yes (but only the sense that it can produce real-valued outputs for actions).


Does it converge?
Yes
Not necessarily
Not necessarily


Is it an online learning algorithm?
Yes
No, if you use experience replay
No, but it can be used in an online learning setting