Hugging Face's logo

LnL-AI
/
dbrx-instruct-tokenizer

Transformers
Transformers.js
tokenizers
Model card Files Community
Qubitium commited on
Commit
0aca37c
·
verified ·
1 Parent(s): 23a0fef

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -5,6 +5,8 @@ tags:
5
  - tokenizers
6
  ---
7
 
 
 
8
  # DBRX Instruct Tokenizer
9
 
10
  A 🤗-compatible version of the **DBRX Instruct** (adapted from [databricks/dbrx-instruct](https://huggingface.co/databricks/dbrx-instruct)). This means it can be used with Hugging Face libraries including [Transformers](https://github.com/huggingface/transformers), [Tokenizers](https://github.com/huggingface/tokenizers), and [Transformers.js](https://github.com/xenova/transformers.js).
 
5
  - tokenizers
6
  ---
7
 
8
+ ## Original Author: @xenova https://huggingface.co/Xenova, repo: https://huggingface.co/Xenova/dbrx-instruct-tokenizer
9
+
10
  # DBRX Instruct Tokenizer
11
 
12
  A 🤗-compatible version of the **DBRX Instruct** (adapted from [databricks/dbrx-instruct](https://huggingface.co/databricks/dbrx-instruct)). This means it can be used with Hugging Face libraries including [Transformers](https://github.com/huggingface/transformers), [Tokenizers](https://github.com/huggingface/tokenizers), and [Transformers.js](https://github.com/xenova/transformers.js).