Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
atiwari751
/
Hindi-tokenizer
like
0
Sleeping
App
Files
Files
Community
Fetching metadata from the HF Docker repository...
c128a5f
Hindi-tokenizer
1 contributor
History:
5 commits
atiwari751
1.4M text 7.53X compression
c128a5f
3 months ago
.gitignore
30 Bytes
1.4M text 7.53X compression
3 months ago
BPE.py
1.91 kB
1.4M text 7.53X compression
3 months ago
README.md
Safe
0 Bytes
added README
3 months ago
data_analysis.py
803 Bytes
Hindi tokenizer 101
3 months ago
decoded_output.txt
2.95 kB
1.4M text 7.53X compression
3 months ago
encode_decode.py
2.67 kB
1.4M text 7.53X compression
3 months ago
encode_input.txt
1.95 kB
1.4M text 7.53X compression
3 months ago
text_file_eng.txt
30 kB
Hindi tokenizer 101
3 months ago
