Spaces:

yhavinga
/

dutch-tokenizer-arena

Running

App Files Files Community

dutch-tokenizer-arena / tokenizer

3 contributors

History: 4 commits

xu-song's picture

fix unicode error: 'unicodeescape' codec can't decode bytes in position 602-608: unknown Unicode character name

bce41d0 about 1 year ago

tiktoken_patch.py

2.51 kB

fix unicode error: 'unicodeescape' codec can't decode bytes in position 602-608: unknown Unicode character name about 1 year ago