What's the best practice for Boltzmann Exploration temperature in RL?
Do you have to use Boltzmann exploration, strictly? There is a modification for Boltzmann exploration called Mellow-max. It, basically, provides an adaptive temperature for Boltzmann exploration.
Here is the link for the paper for tuning mellow-max with deep reinforcement learning (DQN is often mentioned): http://cs.brown.edu/people/gdk/pubs/tuning_mellowmax_drlw.pdf
Here is the link for mellow-max implemented with SARSA (I recommend reading this first, to get an understanding of mellow-max): https://arxiv.org/pdf/1612.05628.pdf