Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
atiwari751
/
Hindi-tokenizer
like
0
Sleeping
App
Files
Files
Community
Fetching metadata from the HF Docker repository...
fa753cb
Hindi-tokenizer
1 contributor
History:
25 commits
atiwari751
removed pkl file to address merge conflict
fa753cb
2 months ago
.gitignore
84 Bytes
added Hindi result files
2 months ago
BPE.py
3.54 kB
removed pkl file to address merge conflict
2 months ago
README.md
Safe
0 Bytes
added README
3 months ago
data_analysis.py
803 Bytes
Hindi tokenizer 101
3 months ago
decoded_output.txt
2.93 kB
Hindi regex cheat code applied
2 months ago
encode_decode.py
4.36 kB
Hindi regex cheat code applied
2 months ago
encode_input.txt
Safe
2.62 kB
Hindi regex brutality
2 months ago
text_file_eng.txt
30 kB
Hindi tokenizer 101
3 months ago
text_file_eng_long.txt
125 kB
english trial long
2 months ago
