mt5-small / tokenizer_config.json

Commit History

FIX special token tokenization errors in MT5 tokenizer
485419b
verified

Dhurmir commited on