How does batch normalization work for convolutional neural networks
Method 2:
 This is original batch Normalization as suggested in the paper [Ioffe &amp; Szegedy, 2015].
It is the most common approach. It is very well explained here [d2l.ai]

Similarly, with convolutional layers, we can apply batch normalization after the convolution and before the nonlinear activation function. When the convolution has multiple output channels, we need to carry out batch normalization for each of the outputs of these channels, and each channel has its own scale and shift parameters, both of which are scalars. Assume that our minibatches contain  m  examples and that for each channel, the output of the convolution has height  p  and width  q . For convolutional layers, we carry out each batch normalization over the  m⋅p⋅q  elements per output channel simultaneously. Thus, we collect the values over all spatial locations when computing the mean and variance and consequently apply the same mean and variance within a given channel to normalize the value at each spatial location


I'm not sure what the authors mean by &quot;per feature map&quot;, does this mean per channel?

Yes, two trainable parameters per Channel/Feature map.
Method 3:
 This was the idea suggested as &quot;Layer Normalization&quot;. [Paper]
 It fixed the issue of BN i.e. dependence on a Batch of data and also it worked for sequence data. But the paper didn't claim anything great for CNN.

We have also experimented with convolutional neural networks. In our preliminary experiments, we observed that layer normalization offers a speedup over the baseline model without normalization, but batch normalization outperforms the other methods. With fully connected layers, all the hidden units in a layer tend to make similar contributions to the final prediction and re-centering and rescaling the summed inputs to a layer works well. However, the assumption of similar contributions
is no longer true for convolutional neural networks. The large number of the hidden units whose receptive fields lie near the boundary of the image are rarely turned on and thus have very different statistics from the rest of the hidden units within the same layer. We think further research is needed to make layer normalization work well in ConvNets

Method 1:
 This is averaging across the Feature Maps on every pixel. I am not sure of its aplication.