What activation functions are currently popular? I am not asking what activation function is better. I want to know what activation functions are more used in research or deployment. Also, are they used in combination? e.g. ReLU, ELUs, etc. I'd appreciate any statistic on this or insight.
Currently both ReLU and ELUs are the most popular activation functions (AF) used in neural nets (NNs). This is because they eliminate the vanishing gradient problem that causes major problems in the training process and degrades the accuracy and performance of NN models.
Also these AFs, more specifically ReLU, are very fast learning AF which make them even more useful in research.
However, depending on type of NN you working on, it's always a good practice to pay some attentions to new studies.