Why did Distributional Q Learning go out of popularity? I read some papers (for example, this) and blogs that spoke about the advantages of distributional Q learning. However, it no longer seems to come up in literature. Did it have any shortcomings that led to its failure? If yes, can someone can talk about it here?
Actually, distributional RL is a well-studied field in deep RL. Generally speaking, distributional RL needs more computing sources (1.1X), because of the quantile head. We can also find new distributional RL literature in NeurIPS2020 :).