Are all 110 million parameter in bert are trainable
All these parameters are trainable.
Note that in normal Transformers it is typical to have fixed (non-trainable) positional embeddings, but in BERT they are learned.
Note also the &quot;pooler&quot; component, which is an extra projection that was not mentioned in the paper, but which the authors commented on later.