What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?
How many classes do you have? Bert can handle a high-quality 12k dataset for binary classification. I recommend duplicating your positive test case 4x and sampling a 5k test cases from your negative class. This will give you a balanced dataset. Then implement BERT in google colab using the original GitHub repository from the google Bert team. You can find a functional collab here: https://colab.research.google.com/github/tensorflow/tpu/blob/master/tools/colab/bert_finetuning_with_cloud_tpus.ipynb
On collab, TPUs takes about 10 to 15 minutes to run. All you have to do is use write your data processor.
The following parameters are a good place to start.

TRAIN_BATCH_SIZE = 32
EVAL_BATCH_SIZE = 8
PREDICT_BATCH_SIZE = 8
LEARNING_RATE = 2e-5
NUM_TRAIN_EPOCHS = 3.0
MAX_SEQ_LENGTH = 128
# Warmup is a period of time where hte learning rate 
# is small and gradually increases--usually helps training.
WARMUP_PROPORTION = 0.1
# Model configs
SAVE_CHECKPOINTS_STEPS = 1000
SAVE_SUMMARY_STEPS = 500