internlm2-chat-7b / tokenization_internlm2_fast.py

Commit History

Fix no white space when using stream_chat with fast tokenizer
065ad54
verified

x54-729 commited on

fast tokenizer and stream_chat fix (#5)
1334f62
verified

RangiLyu x54-729 commited on