monsoon-nlp
/

dna-blockdiff

Model card Files Files and versions Community

dna-blockdiff / tokenizer_config.json

monsoon-nlp's picture

include code and tokenizer

0a77499 4 months ago

129 Bytes

	{
	"clean_up_tokenization_spaces": true,
	"eos_token": null,
	"model_max_length": 1024,
	"tokenizer_class": "EsmTokenizer"
	}