You'll have to look up some research, but here are two reasons off top of my head:
1. State-of-the-art transformers are extremely large in their size (I'm proud of my small-ish-ness)
2. RNNs might be more interpretable due to their inherent simplicity
spam fails_task lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor helpfulness creativity violence