Why isn't there just one "keystone" activation function in Neural Networks?
Those are old articles.  Tinkering with activation functions probably isn't your best use of time, in most cases.  Today, the standard engineering practice is (to a first order of approximation): use ReLU and don't stress over it.  ReLU is clearly superior to sigmoid and tanh for most cases, so if you read older articles they'll talk about sigmoid and tanh, but today, ReLU has replaced them.  There are fancier newer activation functions that in some cases are slightly better than ReLU and in some cases are slightly worse but the short version is ReLU is good enough and don't worry about the others at this stage in your learning and knowledge; just use ReLU and call it a day.
This is a crude simplification and there are absolutely exceptions but I'm giving you a rule of thumb that will be pretty reasonable in practice.
Why?  My main answer is that you will need to get used to the fact that when working with neural networks, we don't really know the answer to most &quot;why&quot; questions.  Sometimes we have intuition and theories but at its heart this is an empirical science: we don't really understand why neural networks work well.  There are papers that give some explanation of why ReLU seems to do better than sigmoid/tanh -- in particular, sigmoid/tanh suffer from vanishing gradients when their inputs are in the tails of the sigmoid/tanh (as then their output is exponentially small, so the gradient is essentially zero), and then training gets stuck or proceeds very slowly -- but don't expect great theory that is going to tell you what to do.  Instead, this is largely an empirical science, and if we're lucky, we have experiments and theory that helps us understand the empirical data we see.
I don't see any reason to expect there to be a single activation function that is optimal for all tasks, so I'm not bothered if that isn't true and don't feel that we need a &quot;reason&quot; for it to be false.