Update README.md
Browse files
README.md
CHANGED
@@ -19,11 +19,11 @@ base_model: mistralai/Mistral-7B-v0.1
|
|
19 |
|
20 |
# Mistral-7B-Instruct-v0.1
|
21 |
|
22 |
-
General instruction-following
|
23 |
|
24 |
## Model Details
|
25 |
|
26 |
-
This instruction-following
|
27 |
|
28 |
- **Developed by:** Daniel Furman
|
29 |
- **Model type:** Decoder-only
|
@@ -154,7 +154,7 @@ You are a helpful assistant.<|im_end|>
|
|
154 |
## Training Hyperparameters
|
155 |
|
156 |
|
157 |
-
We use the [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) from `trl` to fine-tune
|
158 |
|
159 |
The following `TrainingArguments` config was used:
|
160 |
|
|
|
19 |
|
20 |
# Mistral-7B-Instruct-v0.1
|
21 |
|
22 |
+
General instruction-following LLM finetuned from [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1).
|
23 |
|
24 |
## Model Details
|
25 |
|
26 |
+
This instruction-following LLM was built via parameter-efficient QLoRA finetuning of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the first 5k rows of [ehartford/dolphin](https://huggingface.co/datasets/ehartford/dolphin). Finetuning was executed on 1x A100 (40 GB SXM) for roughly 1 hour on Google Colab.
|
27 |
|
28 |
- **Developed by:** Daniel Furman
|
29 |
- **Model type:** Decoder-only
|
|
|
154 |
## Training Hyperparameters
|
155 |
|
156 |
|
157 |
+
We use the [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) from `trl` to fine-tune LLMs on instruction-following datasets.
|
158 |
|
159 |
The following `TrainingArguments` config was used:
|
160 |
|