Update README.md

README.md

@@ -6,9 +6,6 @@ language:
 - 'no'
 - en
 pipeline_tag: text-generation
-inference:
-  parameters:
-    temperature: 0.7
 tags:
 - translation
 ---
@@ -21,53 +18,17 @@ Translate text data from English to Swedish, or Swedish to English.
 
 
 ## How to use:
-```python
-import torch
-from transformers import pipeline, StoppingCriteriaList, StoppingCriteria
-
-device = "cuda" if torch.cuda.is_available() else "cpu"
-
-
-# (Optional) - define a stopping criteria
-# We ideally want the model to stop generate once the response from the Bot is generated
-class StopOnTokenCriteria(StoppingCriteria):
-    def __init__(self, stop_token_id):
-        self.stop_token_id = stop_token_id
-
-    def __call__(self, input_ids, scores, **kwargs):
-        return input_ids[0, -1] == self.stop_token_id
-
-
-pipe = pipeline(
-    task="text-generation",
-    model="AI-Sweden-Models/gpt-sw3-6.7b-v2-translator",
-    device=device
-)
-
-stop_on_token_criteria = StopOnTokenCriteria(stop_token_id=pipe.tokenizer.bos_token_id)
-text = "I like to eat ice cream in the summer."
-
-# This will translate English to Swedish
-# To translate from Swedish to English the prompt would be:
-# prompt = f"<|endoftext|><s>User: Översätt till Engelska från Svenska\n{text}<s>Bot:"
-
-prompt = f"<|endoftext|><s>User: Översätt till Svenska från Engelska\n{text}<s>Bot:"
-
-input_tokens = pipe.tokenizer(prompt, return_tensors="pt").input_ids.to(device)
-max_model_length = 2048
-dynamic_max_length = max_model_length - input_tokens.shape[1]
-
-response = pipe(
-    prompt,
-    max_length=dynamic_max_length,
-    truncation=True,
-    stopping_criteria=StoppingCriteriaList([stop_on_token_criteria])
-)
-
-print(response[0]["generated_text"].split("<s>Bot: ")[-1])
+Swedish to English:
+```bash
+FROM ./gpt-sw3-6-7b-v2-translator-SIZE.gguf
+TEMPLATE """"<|endoftext|><s>User: Översätt till Svenska från Engelska\n{{ .Prompt }}<s>Bot:""""
+PARAMETER stop <s>
 ```
-```python
->>> "Jag tycker om att äta glass på sommaren."
+English to Swedish:
+```bash
+FROM ./gpt-sw3-6-7b-v2-translator-SIZE.gguf
+TEMPLATE """"<|endoftext|><s>User: Översätt till Engelska från Svenska\n{{ .Prompt }}<s>Bot:""""
+PARAMETER stop <s>
 ```
 
 ## Training & Data: