GGUF
English
mpt
code
Composer
MosaicML
llm-foundry
StreamingDatasets
custom_code
modularbot commited on
Commit
e0686f2
·
verified ·
1 Parent(s): 36b5bd1

Added chat template to tokenizer_config.json

Browse files
Files changed (1) hide show
  1. tokenizer_config.json +2 -1
tokenizer_config.json CHANGED
@@ -19,9 +19,10 @@
19
  "<extra_id_10>"
20
  ],
21
  "bos_token": "<|endoftext|>",
 
22
  "clean_up_tokenization_spaces": false,
23
  "eos_token": "<|endoftext|>",
24
  "model_max_length": 4096,
25
  "tokenizer_class": "GPTNeoXTokenizer",
26
  "unk_token": "<|endoftext|>"
27
- }
 
19
  "<extra_id_10>"
20
  ],
21
  "bos_token": "<|endoftext|>",
22
+ "chat_template": "{{ .Prompt }}",
23
  "clean_up_tokenization_spaces": false,
24
  "eos_token": "<|endoftext|>",
25
  "model_max_length": 4096,
26
  "tokenizer_class": "GPTNeoXTokenizer",
27
  "unk_token": "<|endoftext|>"
28
+ }