Smooth Q-learning: Accelerate Convergence of Q-learning Using Similarity
  An improvement of Q-learning is proposed in this paper. It is different from
classic Q-learning in that the similarity between different states and actions
is considered in the proposed method. During the training, a new updating
mechanism is used, in which the Q value of the similar state-action pairs are
updated synchronously. The proposed method can be used in combination with both
tabular Q-learning function and deep Q-learning. And the results of numerical
examples illustrate that compared to the classic Q-learning, the proposed
method has a significantly better performance.
