Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up

Spaces:

atiwari751
/

Hindi-tokenizer

Sleeping

App Files Files Community

Fetching metadata from the HF Docker repository...

Hindi-tokenizer

1 contributor

History: 6 commits

atiwari751's picture

basic regex

1e8ebcb 3 months ago

.gitignore
30 Bytes

1.4M text 7.53X compression 3 months ago
BPE.py
2.17 kB

basic regex 3 months ago
README.md

0 Bytes

added README 3 months ago
data_analysis.py
803 Bytes

Hindi tokenizer 101 3 months ago
decoded_output.txt
32 Bytes

basic regex 3 months ago
encode_decode.py
1.49 kB

basic regex 3 months ago
encode_input.txt
1.95 kB

1.4M text 7.53X compression 3 months ago
text_file_eng.txt
30 kB

Hindi tokenizer 101 3 months ago