Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
stabilityai
/
arcade100k
like
3
Follow
Stability AI
28.2k
arxiv:
2402.17834
Model card
Files
Files and versions
xet
Community
ee2a4ae
arcade100k
Ctrl+K
Ctrl+K
2 contributors
History:
17 commits
jon-tow
feat: expose tiktoken tokenizer merge ranks and special tokens
ee2a4ae
over 1 year ago
.gitattributes
Safe
1.52 kB
initial commit
almost 2 years ago
LICENSE
6.9 kB
fix: update license
over 1 year ago
README.md
258 Bytes
Update README.md
almost 2 years ago
arcade100k.tiktoken
Safe
1.68 MB
fix: store `cl100k_base` mergeable ranks in `arcade100k.tiktoken`
over 1 year ago
tokenization_arcade100k.py
9.46 kB
feat: expose tiktoken tokenizer merge ranks and special tokens
over 1 year ago
tokenizer_config.json
160 Bytes
Create tokenizer_config.json
over 1 year ago