Is there a mathematical proof that shows that certain parameters work "better" than others for a certain task?
There is stuff like the Universal Approximation Theorem. 

There are also investigations into the loss surface of neural networks.

And classics like this explanation of the vanishing gradient problem.

But I'm afraid the mathematical theory of neural networks only exists in bits and pieces in many different papers. And many of the most important questions can currently only be answered empirically.