How does layer normalization work exactly?
To understand how it works, we can have a look at the paper related to the question: &quot;Layer Normalization&quot;, J. Lei, J. Kiros and G. Hinton.

Question: layer normalization normalizes across all the features for fully connected layers

Not exactly. What layer normalization does is to compute the normalization of the term $a_i^l$ of each neuron $i$ of the layer $l$ within the layer (and not across all the features or activations of the fully connected layers).
This term $a_i^l$  is given by the weighted sum of the activations of the previous layers:
$$ a_i^l = (w_i^l)^T\, h^l$$
Where $h^l$ is the vector of activations from the previous layer and $w_i^l$ is the vector of weights that connects the activations of the previous layer with the neuron $i$.
This normalization is computed within the layer because is done by using the mean ($\mu^l$) and standard deviation ($\sigma^l$) of the values of all the neurons in the same layer, $l$ for the current sample:
$$ \mu^l = \frac{1}{H}\sum_{i=1}^Ha_i^l\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\sigma^l=\sqrt{\frac{1}{H}\sum_{i=1}^H(a_i^l-\mu^l)^2}$$
Where $H$ is the number of neurons in the layer $l$. Hence, the normalization of the $a^l_i$ terms is given by:
$$\bar{a}_i^l = \frac{g_i^l}{\sigma^l}(a_i^l -\mu^l)$$
Where $g^i_l$ is defined by the authors as a scale (or gain) value for each neuron $i$ of the layer $l$ which has to be learnt by the NN.
Once we have obtained the normalized term $\bar{a}_i^l$, we compute the activation of each neuron, $h_i^{l+1}$ by:
$$ h_i^{l+1} = f(\bar{a}_i^l + b_i^l)$$
Where $b_i^l$ is the bias term of the neuron, and $f(\cdot)$ is the activation function.

Question: Does that mean that for each batch dimension we have to learn the normalization parameters?

Yes, that's true. For each sample we would have to recompute the terms $\mu^l$ and $\sigma^l$ in order to do the normalization of the $a^l_i$ terms.

Question: What do we do at test time if our batch dimension is different?

At test time we have to compute only the predictions (forward propagation). We aren't learning the weights, biases and the $g_i$'s terms, we are testing them. However, because the activations on each layer have to be normalized we have to compute again the terms $\mu^l$ and $\sigma^l$ for each layer and sample (irrespective of the mini-batch size).