We use the pre-trained uncased BERT$_\mathrm {BASE}$ model for fine-tuning, because we find that BERT$_\mathrm {LARGE}$ model performs slightly worse than BERT$_\mathrm {BASE}$ in this task.  
 Question: Do they use large or small BERT?
small BERT