Batch norm: why the initial normalization? I'm a beginner in NNs and the first thing I don't understand with batch norm is the following 2 steps:


First we normalize the batch data on a z parameter to Mu=0, sigma^2=1
Then we change z via the coefficients of Mu, sigma^2 (usu. alpha, beta) by updating them as learnable parameters.


I don't understand why the first step is necessary if we change the distribution in the second step anyway. Could someone explain please?
I hope I got your question correctly...

It’s called “Batch” Normalization because we perform this transformation and calculate the statistics only for a subpart (a batch) of the entire training set not as a whole.