Is there any activation function that can make batch normalization obsolete?
What you describe sounds a lot like Scaled-Exponential Linear Units (SELUs), that are the core of Self-Normalizing Neural Networks, which where presented at NIPS 2017.

A short summary from here is that:


  If the mean and variance of the input is in certain range, then the mean and variance of the output should (1) also in that range and (2) converge to a fixed point after iteratively applying the activation function.


You might want to have a look at the reddit post comments. If you want to fully understand them, you can go ahead with the 90 page-long appendix of the arxiv preprint.

They got a lot of attention when they were presented, but I think they have not delivered up to the expectations, as no one seems to be talking about them lately on the internet.