We use the following subset of GLUE tasks BIBREF4 for fine-tuning:

MRPC: the Microsoft Research Paraphrase Corpus BIBREF13

STS-B: the Semantic Textual Similarity Benchmark BIBREF14

SST-2: the Stanford Sentiment Treebank, two-way classification BIBREF15

QQP: the Quora Question Pairs dataset

RTE: the Recognizing Textual Entailment datasets

QNLI: Question-answering NLI based on the Stanford Question Answering Dataset BIBREF3

MNLI: the Multi-Genre Natural Language Inference Corpus, matched section BIBREF16 
 Question: What subset of GLUE tasks is used?
MRPC STS-B SST-2 QQP RTE QNLI MNLI