Update README.md

README.md

@@ -50,6 +50,7 @@ It took ~1 hour to train 1 epoch on 1x A100.
 
 Prompt format:
 This model (and all my future releases) use [ChatML](https://huggingface.co/docs/transformers/chat_templating#what-template-should-i-use) prompt format.
+
 ```
 <|im_start|>system
 You are a helpful assistant.<|im_end|>
@@ -57,6 +58,7 @@ You are a helpful assistant.<|im_end|>
 {prompt}<|im_end|>
 <|im_start|>assistant
 ### Training Hyperparameters
+```
 
 We use the [`SFTTrainer`] (https://huggingface.co/docs/trl/main/en/sft_trainer) from 🤗's TRL package to easily fine-tune llms on instruction-following datasets.
 
@@ -91,7 +93,84 @@ The following `bitsandbytes` quantization config was used:
 
 Use the code below to get started with the model.
 
-[More Information Needed]
+```python
+!pip install -q -U transformers peft torch accelerate bitsandbytes einops sentencepiece
+
+import torch
+from peft import PeftModel, PeftConfig
+from transformers import (
+    AutoModelForCausalLM,
+    AutoTokenizer,
+    BitsAndBytesConfig,
+)
+```
+
+```python
+peft_model_id = "dfurman/mistral-7b-instruct-v0.1"
+config = PeftConfig.from_pretrained(peft_model_id)
+
+model = AutoModelForCausalLM.from_pretrained(
+    config.base_model_name_or_path,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+    trust_remote_code=True,
+)
+
+tokenizer = AutoTokenizer.from_pretrained(
+    config.base_model_name_or_path,
+    use_fast=True, 
+    trust_remote_code=True,
+)
+
+model = PeftModel.from_pretrained(model, peft_model_id)
+
+format_template = "You are a helpful assistant. Write a response that appropriately completes the request. {query}\n"
+```
+
+```python
+query = "Write a short email inviting my friends to a dinner party on Friday. Respond succinctly."
+prompt = format_template.format(query=query)
+
+input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()
+with torch.autocast("cuda", dtype=torch.bfloat16):
+    output = model.generate(
+        input_ids=input_ids,
+        max_new_tokens=512,
+        do_sample=True,
+        temperature=0.1,
+        return_dict_in_generate=True,
+        eos_token_id=tokenizer.eos_token_id,
+        pad_token_id=tokenizer.pad_token_id,
+        repetition_penalty=1.2,
+        no_repeat_ngram_size=5,
+    )
+
+print("\n\n*** Generate:")
+print(tokenizer.decode(output["sequences"][0][len(input_ids[0]):], skip_special_tokens=True))
+```
+
+<details>
+
+<summary>Output</summary>
+
+**Prompt**: Write a short email inviting my friends to a dinner party on Friday. Respond succinctly.
+
+**Generation**: The invitation should be brief and to-the-point, so it's best to use simple language and avoid unnecessary details or long explanations. Here is an example of a concise invitation:
+
+Dear Friends,
+
+I hope you can join me for a fun evening at my place this Friday! We'll have delicious food, great conversation, and maybe even some games if we feel like it. Please RSVP by Wednesday night so I know who will be there. 
+
+Looking forward to seeing you all soon!
+
+Best regards,
+Your Name
+
+This message clearly communicates the essential information about the event while maintaining a friendly tone. It also includes a specific date (Friday) and timeframe (evening), as well as a clear call to action (RSVP). The closing line adds a personal touch and expresses excitement for the gathering. Overall, this invitation strikes a good balance between being informative and engaging without overwhelming the reader with too much text.
+
+Remember, when writing emails, always keep in mind your audience and their preferences. If they prefer more detailed information or additional context, adjust accordingly. However, try not to make the invitation overly complicated or lengthy – simplicity often makes for a better experience. Happy emailing!
+
+</details>
 
 ### Speeds, Sizes, Times