title: Batch normalization during testing
body: <p>For batch normalization during testing, how does one calculate the mean and variance of each activation input (in each layer and input dimension)? Does one record the means and variances from training, calculate the means and variances of the entire training set, or calculate the means and variances of the entire test set?</p>
  
  <p>Many people say you have to precalculate the means and variances, but if you use the method of calculating the means and variances of the entire test set, wouldn't you need to calculate the means and variances of the entire test set while performing forward propagation (not "pre")?</p>
  
  <p>Thank you so much for all your help!</p>