add tokenizer config from perplexity metric. truncation breaks tests 4bd2962 jgauthier commited on Jul 7, 2022