jlzhou commited on
Commit
a1bf35e
·
verified ·
1 Parent(s): 80e1091

Update README.md

Browse files

Add quickstart code

Files changed (1) hide show
  1. README.md +34 -1
README.md CHANGED
@@ -68,7 +68,40 @@ Users (both direct and downstream) should be made aware of the risks, biases and
68
 
69
  Use the code below to get started with the model.
70
 
71
- [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
72
 
73
  ## Training Details
74
 
 
68
 
69
  Use the code below to get started with the model.
70
 
71
+ ```python
72
+ from transformers import AutoModelForCausalLM, AutoTokenizer
73
+
74
+ model_name = "jlzhou/Qwen2.5-3B-Infinity-Instruct-0625"
75
+
76
+ model = AutoModelForCausalLM.from_pretrained(
77
+ model_name,
78
+ torch_dtype="auto",
79
+ device_map="auto"
80
+ )
81
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
82
+
83
+ prompt = "Give me a short introduction to large language model."
84
+ messages = [
85
+ {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},
86
+ {"role": "user", "content": prompt}
87
+ ]
88
+ text = tokenizer.apply_chat_template(
89
+ messages,
90
+ tokenize=False,
91
+ add_generation_prompt=True
92
+ )
93
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
94
+
95
+ generated_ids = model.generate(
96
+ **model_inputs,
97
+ max_new_tokens=512
98
+ )
99
+ generated_ids = [
100
+ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
101
+ ]
102
+
103
+ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
104
+ ```
105
 
106
  ## Training Details
107