After having read matus's beautiful answer in this thread explaining (among other things) Kolmogorov's result regarding the Universal Approximation Theorem with Neural Networks, I wonder: if just $\mathcal{O}$($d^2$) nodes can replicate (!) the target function, provided we can use different transfer functions, do there exist theory or algorithms for inferring from the training data which transfer functions to use?
Practical Implications of Kolmogorov's Result on the Universal Approximation Theorem with Neural Networks
approximation algorithms;machine learning;ne.neural evol;st.statistics