problem with chinese prompt

#2
by cmgzy - opened

image.png

It seems that tokenizer does not work correctly to process chinese characters. The prompt in chinese means "5.9 and 5.11, which one is bigger?"
The model did recognize the numbers, 5.9 and 5.11, but did not figure out the meaning of prompt.

cmgzy changed discussion status to closed

Sign up or log in to comment