Update README.md
Browse files
README.md
CHANGED
@@ -166,8 +166,16 @@ Shake all ingredients in a shaker filled with ice until well chilled and strain
|
|
166 |
|
167 |
It took ~3 hours to train 3 epochs on 1x A100 (40 GB SXM).
|
168 |
|
169 |
-
Prompt
|
170 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
171 |
|
172 |
```python
|
173 |
messages = [
|
@@ -188,7 +196,7 @@ print(prompt)
|
|
188 |
```
|
189 |
</details>
|
190 |
|
191 |
-
|
192 |
|
193 |
|
194 |
We use the [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) from `trl` to fine-tune LLMs on instruction-following datasets.
|
|
|
166 |
|
167 |
It took ~3 hours to train 3 epochs on 1x A100 (40 GB SXM).
|
168 |
|
169 |
+
### Prompt Format
|
170 |
+
|
171 |
+
This model was finetuned with the following format:
|
172 |
+
|
173 |
+
```python
|
174 |
+
tokenizer.chat_template = "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ ' [INST] ' + message['content'] + ' [/INST] ' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token + ' ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}"
|
175 |
+
```
|
176 |
+
|
177 |
+
|
178 |
+
This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method. Here's an illustrative example:
|
179 |
|
180 |
```python
|
181 |
messages = [
|
|
|
196 |
```
|
197 |
</details>
|
198 |
|
199 |
+
### Training Hyperparameters
|
200 |
|
201 |
|
202 |
We use the [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) from `trl` to fine-tune LLMs on instruction-following datasets.
|