Commit History
Update decode method in tokenizer
4d0fc39
duzx16
commited on
Always add gmask in token ids
3a99d79
duzx16
commited on
Fix bug
53f0197
duzx16
commited on
Fix eos token in tokenizer
aa51e62
duzx16
commited on
Fix tokenizer config saving
7e69b85
duzx16
commited on
Add gmask token id
36b7f2d
duzx16
commited on
Update code for slim
63ce1ba
duzx16
commited on
Drop icetk dependency
72985e8
duzx16
commited on
Fix decode method for torch tensor
23ad39b
duzx16
commited on
Fix generate
fb23542
duzx16
commited on
Fix attention_mask and position_ids
373fd6b
duzx16
commited on
Fix encode method
e22cddf
duzx16
commited on
Fix batch input
e1494f2
duzx16
commited on
Implement batch generation
cc96a22
duzx16
commited on
Change padding side
db22499
duzx16
commited on
Fix typo in tokenization_chatglm.py
1b54948
songxxzp
commited on
Remove image tokens when decoding
8492687
duzx16
commited on
Update tokenizer
c4575e7
duzx16
commited on
