Commit
·
ba61d13
1
Parent(s):
9c18b58
adding model card
Browse files
README.md
CHANGED
@@ -4,12 +4,13 @@ Introducing TenyxChat, a series of ChatGPT-like models trained to function as us
|
|
4 |
|
5 |
We fine-tune [Openchat-3.5](https://arxiv.org/pdf/2309.11235.pdf) with our proprietary approach ([blog](https://www.tenyx.com/post/forgetting-and-toxicity-in-llms-a-deep-dive-on-fine-tuning-methods), [service](https://www.tenyx.com/fine-tuning)), which shows an increase in [MT-Bench](https://arxiv.org/abs/2306.05685), without a drop in performance of the model on other benchmarks. Our approach aims to mitigate forgetting in LLMs in a computationally efficient manner, thereby enabling continual fine-tuning capabilities without altering the pre-trained output distribution. TenyxChat-7B-v1 was trained using eight A100s (80GB) for two hours, with a training setup obtained from HuggingFaceH4 ([GitHub](https://github.com/huggingface/alignment-handbook)).
|
6 |
|
7 |
-
|
|
|
|
|
|
|
|
|
|
|
8 |
|
9 |
-
- Model type: Fine-tuned 7B model for chat.
|
10 |
-
- License: Apache 2.0
|
11 |
-
- Base model: OpenChat 3.5 ([https://huggingface.co/openchat/openchat_3.5](https://huggingface.co/openchat/openchat_3.5))
|
12 |
-
- Demo: Hugging face space
|
13 |
|
14 |
## Usage
|
15 |
|
|
|
4 |
|
5 |
We fine-tune [Openchat-3.5](https://arxiv.org/pdf/2309.11235.pdf) with our proprietary approach ([blog](https://www.tenyx.com/post/forgetting-and-toxicity-in-llms-a-deep-dive-on-fine-tuning-methods), [service](https://www.tenyx.com/fine-tuning)), which shows an increase in [MT-Bench](https://arxiv.org/abs/2306.05685), without a drop in performance of the model on other benchmarks. Our approach aims to mitigate forgetting in LLMs in a computationally efficient manner, thereby enabling continual fine-tuning capabilities without altering the pre-trained output distribution. TenyxChat-7B-v1 was trained using eight A100s (80GB) for two hours, with a training setup obtained from HuggingFaceH4 ([GitHub](https://github.com/huggingface/alignment-handbook)).
|
6 |
|
7 |
+
---
|
8 |
+
model_type: Fine-tuned 7B model for chat.
|
9 |
+
license: {apache-2.0}
|
10 |
+
base_model: {openchat/openchat_3.5}
|
11 |
+
demo: [Hugging Face Spaces](https://huggingface.co/spaces/tenyx/TenyxChat-7B-v1)
|
12 |
+
---
|
13 |
|
|
|
|
|
|
|
|
|
14 |
|
15 |
## Usage
|
16 |
|