sentence1: bert-ft is a pre-trained bidirectional transformer language model following a fine-tuning step on cosmos qa .
sentence2: bert employs the recently proposed transformer layers in conjunction with a masked language modeling objective as a pre-trained sentence encoder .