Update README.md

README.md

@@ -1,3 +1,97 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+base_model:
+- cyberagent/DeepSeek-R1-Distill-Qwen-14B-Japanese
+datasets:
+- HuggingFaceH4/ultrachat_200k
+---
+
+# TinyDeepSeek-JP-1.5B
+
+本モデルは, DeepSeek-R1の小型蒸留モデルに日本語を追加学習した[cyberagent/DeepSeek-R1-Distill-Qwen-14B-Japanese](https://huggingface.co/cyberagent/DeepSeek-R1-Distill-Qwen-14B-Japanese)に対し、
+SakanaAI社が提案した新たな蒸留手法TAIDを適用して小型化したものです.
+
+Teacher model : [cyberagent/DeepSeek-R1-Distill-Qwen-14B-Japanese](https://huggingface.co/cyberagent/DeepSeek-R1-Distill-Qwen-14B-Japanese)
+Student model : [SakanaAI/TinySwallow-1.5B-Instruct](https://huggingface.co/SakanaAI/TinySwallow-1.5B-Instruct)
+
+
+### Uses
+Uses follow the original models.    
+This model is provided for research and development purposes only and should be considered as an experimental prototype. It is not intended for commercial use or deployment in mission-critical environments. Use of this model is at the user's own risk, and its performance and outcomes are not guaranteed. EQUES Inc. shall not be liable for any direct, indirect, special, incidental, or consequential damages, or any loss arising from the use of this model, regardless of the results obtained. Users must fully understand the risks associated with the use of this model and use it at their own discretion.
+
+
+### Output Examples
+
+<details><summary>Give me a short introduction to large language model.</summary>
+  
+```
+
+```
+
+</details>
+
+
+<details><summary>大規模言語モデルについて教えて。</summary>
+  
+```
+
+```
+
+</details>
+
+<details><summary>A regular hexagon can be divided into six equilateral triangles. If the perimeter of one of the triangles is 21 inches, what is the perimeter, in inches, of the regular hexagon?</summary>
+  
+```
+
+```
+
+</details>
+
+
+
+### Sample Usage
+```python
+import os
+os.environ["CUDA_VISIBLE_DEVICES"]="0"
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "EQUES/TinyDeepSeek-JP-1.5B"
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+prompt = "大規模言語モデルについて教えて。"
+messages = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+
+generated_ids = model.generate(
+    **model_inputs,
+    max_new_tokens=512,
+)
+generated_ids = [
+    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
+]
+
+response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+print(response)
+```
+
+### License
+Apache-2.0
+
+
+### Acknowledgement
+- SakanaAI & Swallow team : development and release of TinySwallow-1.5B
+- SakanaAI : development of TAID
+- CyberAgent : development of DeepSeek-R1-Distill-Qwen-14B-Japanese
+