In our experiments, we compared about 60 elementary metrics, which can be organised as follows:

MT metrics

BLEU, ROUGE, METEOR, TERp

Variants of BLEU: BLEU_1gram, BLEU_2gram, BLEU_3gram, BLEU_4gram and seven smoothing methods from NLTK BIBREF32 .

Intermediate components of TERp inspired by BIBREF18 : e.g. number of insertions, deletions, shifts...

Readability metrics and other sentence-level features: FKGL and FRE, numbers of words, characters, syllables...

Metrics based on the baseline QuEst features (17 features) BIBREF28 , such as statistics on the number of words, word lengths, language model probability and INLINEFORM0 -gram frequency.

Metrics based on other features: frequency table position, concreteness as extracted from BIBREF33 's BIBREF33 list, language model probability of words using a convolutional sequence to sequence model from BIBREF34 , comparison methods using pre-trained fastText word embeddings BIBREF35 or Skip-thought sentence embeddings BIBREF36 . 
 Question: what approaches are compared?
MT metrics Readability metrics and other sentence-level features Metrics based on the baseline QuEst features Metrics based on other features