title: When to use layernorm/batch norm?
body: <p>Where should you splice the normalization when designing a network? E.g. if you have a stacked Transformer or Attention network, does it make sense to normalize any time after you have a dense layer?</p>