Reinforcement Learning in 2018, best tips and tricks? Putting aside things applicable to neural networks such as dropout, l2 regularization, new opitmizers - what are the cool things I should be adding to my Reinforcement Learning algorithm (Q-Learning and SARSA) to make it stronger, in 2018?

So far, I know these:


Experience Replay (gather experiences, then train at once on some of them)
Slowly making our network approach the Q-values of a Target Network. The target Network is only cloned sometimes.
If our function approximator is RNN and our Memory Bank stores sequences of several timesteps, after selecting few random sequences from Memory bank, only backpropagate a half of the sequence, without going back to its start
Have you looked at the Rainbow RL? It combines all improvements of RL. Apart from structural changes other improvements come from Reward Shaping, Curiosity-driven Learning, RL with Unsupervised Auxiliary tasks and RL with external memory (e.g. Neural Episodic Control, Model-free episodic Control). It's pitty that you leave out from your question Policy Gradients, they are lots of fun :) Happy reading!