helenai commited on
Commit
b7ed62f
·
verified ·
1 Parent(s): 19aade4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -1,4 +1,4 @@
1
- This is the [Deepseek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) model, convert to OpenVINO with INT4 weight compression.
2
 
3
  To run inference on this model, install openvino-genai (`pip install openvino-genai`) and run [llm_chat_deepseek.py(https://gist.github.com/helena-intel/554fba91f380df590ecc9245abdad33f)
4
 
@@ -13,4 +13,6 @@ python llm_chat_deepseek.py DeepSeek-R1-Distill-Qwen-7B-ov-int4 GPU
13
  ```
14
 
15
  > [!NOTE]
16
- > The last line specifies the device to run inference. GPU is recommended for recent Intel laptops with integrated graphics, or for Intel discrete graphics. Change to CPU if you do not have an Intel GPU.
 
 
 
1
+ This is the [Deepseek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) model, convert to OpenVINO with INT4 weight compression. This model is optimized for CPU and GPU. See [helenai/DeepSeek-R1-Distill-Qwen-7B-ov-int4-npu](https://huggingface.co/helenai/DeepSeek-R1-Distill-Qwen-7B-ov-int4-npu) for a version that works on NPU.
2
 
3
  To run inference on this model, install openvino-genai (`pip install openvino-genai`) and run [llm_chat_deepseek.py(https://gist.github.com/helena-intel/554fba91f380df590ecc9245abdad33f)
4
 
 
13
  ```
14
 
15
  > [!NOTE]
16
+ > The last line specifies the device to run inference. GPU is recommended for recent Intel laptops with integrated graphics, or for Intel discrete graphics. Change to CPU if you do not have an Intel GPU.
17
+
18
+ Gradio chatbot notebook using this model: https://gist.github.com/helena-intel/69e1c2921a2bcb618fdd7cdfb0bd0202