I think your dataset is too small and your model is too large for the task. 30 examples are not enough to train a model with 128B parameters, even with dropout. You are likely to over-fit the data and have poor generalization performance. I would suggest you either increase your dataset size or reduce your model size. Dropout rate is not the main issue here, but if you want to try it, you can start with a rate of 0.1 and increase it gradually until you see improvement in your validation or test metrics.
spam fails_task lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor helpfulness creativity violence