Hugging Face's logo

Spaces:
atiwari751
/
Hindi-tokenizer
Sleeping

App Files Community
atiwari751 commited on
Commit
610e365
·
1 Parent(s): ee77aba

README updated for HF -2

Browse files
Files changed (1) hide show
  1. README.md +10 -2
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  title: Hindi BPE Tokenizer
3
- emoji: 🇮🇳
4
- colorFrom: orange
5
  colorTo: green
6
  sdk: streamlit
7
  sdk_version: 1.32.0
@@ -12,6 +12,14 @@ pinned: false
12
  # Hindi BPE Tokenizer
13
  A Byte-Pair Encoding tokenizer for Hindi text, implemented using Streamlit.
14
 
 
 
 
 
 
 
 
 
15
  ## Features
16
  - Tokenizes Hindi text using BPE algorithm
17
  - Visualizes the tokenization process
 
1
  ---
2
  title: Hindi BPE Tokenizer
3
+ emoji: 🔤
4
+ colorFrom: blue
5
  colorTo: green
6
  sdk: streamlit
7
  sdk_version: 1.32.0
 
12
  # Hindi BPE Tokenizer
13
  A Byte-Pair Encoding tokenizer for Hindi text, implemented using Streamlit.
14
 
15
+ ## Features
16
+ - Tokenizes Hindi text using BPE algorithm
17
+ - Visualizes the tokenization process
18
+ - Supports custom vocabulary
19
+
20
+ # Hindi BPE Tokenizer
21
+ A Byte-Pair Encoding tokenizer for Hindi text, implemented using Streamlit.
22
+
23
  ## Features
24
  - Tokenizes Hindi text using BPE algorithm
25
  - Visualizes the tokenization process