Why does Batch Normalization work?
To some extend, it get rid of low intensity numerical noise. Condition properties of the optimization problem is always an issue, i suspect BatchNorm alleviate this instability.