monsoon-nlp
/

dna-blockdiff

Model card Files Files and versions Community

dna-blockdiff / tokenizer_config.json

monsoon-nlp's picture

eos

8158986 about 1 month ago

history blame contribute delete

132 Bytes

	{
	"clean_up_tokenization_spaces": true,
	"eos_token": "<eos>",
	"model_max_length": 1024,
	"tokenizer_class": "EsmTokenizer"
	}