Models
Datasets
Spaces
Docs
Enterprise
Pricing
Log In
Sign Up

Spaces:

atiwari751
/

Hindi-tokenizer

Sleeping

App Files Files Community

Fetching metadata from the HF Docker repository...

Hindi-tokenizer

Ctrl+K

Ctrl+K

1 contributor

History: 9 commits

atiwari751's picture

Regex working

76f084f 7 months ago

.gitignore
45 Bytes

regex on byte sequences 7 months ago
BPE.py
3.21 kB

Regex working 7 months ago
README.md

0 Bytes

added README 7 months ago
data_analysis.py
803 Bytes

Hindi tokenizer 101 7 months ago
decoded_output.txt
156 Bytes

Regex working 7 months ago
encode_decode.py
2.36 kB

Regex working 7 months ago
encode_input.txt
117 Bytes

Regex working 7 months ago
text_file_eng.txt
30 kB

Hindi tokenizer 101 7 months ago
text_file_eng_short.txt
272 Bytes

Regex working 7 months ago