dfurman commited on
Commit
b7eea4b
·
1 Parent(s): 12df6af

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -3
README.md CHANGED
@@ -166,8 +166,16 @@ Shake all ingredients in a shaker filled with ice until well chilled and strain
166
 
167
  It took ~3 hours to train 3 epochs on 1x A100 (40 GB SXM).
168
 
169
- Prompt format:
170
- This model uses the same prompt format as [mistralai/Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) and does **not** expect a system prompt. This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method. Here's an illustrative example:
 
 
 
 
 
 
 
 
171
 
172
  ```python
173
  messages = [
@@ -188,7 +196,7 @@ print(prompt)
188
  ```
189
  </details>
190
 
191
- ## Training Hyperparameters
192
 
193
 
194
  We use the [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) from `trl` to fine-tune LLMs on instruction-following datasets.
 
166
 
167
  It took ~3 hours to train 3 epochs on 1x A100 (40 GB SXM).
168
 
169
+ ### Prompt Format
170
+
171
+ This model was finetuned with the following format:
172
+
173
+ ```python
174
+ tokenizer.chat_template = "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ ' [INST] ' + message['content'] + ' [/INST] ' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token + ' ' }}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}"
175
+ ```
176
+
177
+
178
+ This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method. Here's an illustrative example:
179
 
180
  ```python
181
  messages = [
 
196
  ```
197
  </details>
198
 
199
+ ### Training Hyperparameters
200
 
201
 
202
  We use the [SFTTrainer](https://huggingface.co/docs/trl/main/en/sft_trainer) from `trl` to fine-tune LLMs on instruction-following datasets.