Dueling DQN - Advantage Stream, why use average and not the tanh?
The goal of these models is to estimate the value of each action choice. They chose to use the average function to estimate the update value because the average function produces a single scalar influenced by each value. 

Tanh function is not an appropriate nonlinear activation function. The tanh function only takes a scalar as an input, thus would not weight each value. 

The authors tried an appropriate nonlinear activation function for many values - softmax:


  We also experimented with a softmax version of equation (8), but found it to deliver similar results to the simpler module of equation (9).