Does BERT has any advantage over GPT3?
This article on Medium introduces GPT-3 makes some comparisons with BERT.
Specifically, section 4 examines how GPT-3 and BERT differ and mentions that: &quot;On the Architecture dimension, BERT still holds the edge. It’ s trained-on challenges which are better able to capture the latent relationship between text in different problem contexts.&quot;
Also, in section 6 from the article, author lists areas where GPT-3 struggles. It may be that BERT and other bi-directional encoder/transformers may do better, although I have no data/references to support this yet.