InduwaraR commited on
Commit
c156228
·
verified ·
1 Parent(s): 1ae477d

Create README.md (#1)

Browse files

- Create README.md (8430b5ad6a2608639e3e3bd13e192ec0d886bb12)

Files changed (1) hide show
  1. README.md +54 -0
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ base_model:
6
+ - Qwen/Qwen2.5-3B-Instruct
7
+ pipeline_tag: question-answering
8
+ ---
9
+ # Qwen AI Research QA Model (Q4_K_M GGUF)
10
+
11
+ ## Model Overview
12
+ The **Qwen AI Research QA Model** is designed for answering research-oriented AI questions with a focus on precision and depth. This model is optimized in the `Q4_K_M` format for efficient inference while maintaining high-quality responses.
13
+
14
+ ## How to Use
15
+ To use this model with `llama-cpp-python`, follow these steps:
16
+
17
+ ### Installation
18
+ Make sure you have `llama-cpp-python` installed:
19
+ ```bash
20
+ pip install llama-cpp-python
21
+ ```
22
+
23
+ ### Loading the Model
24
+ ```python
25
+ from llama_cpp import Llama
26
+
27
+ llm = Llama.from_pretrained(
28
+ repo_id="InduwaraR/qwen-ai-research-qa-q4_k_m.gguf",
29
+ filename="qwen-ai-research-qa-q4_k_m.gguf",
30
+ )
31
+ ```
32
+
33
+ ### Generating a Response
34
+ ```python
35
+ response = llm.create_chat_completion(
36
+ messages=[
37
+ {"role": "user", "content": "What are the latest advancements in AI research?"}
38
+ ]
39
+ )
40
+ print(response)
41
+ ```
42
+
43
+ ## Model Details
44
+ - **Model Name**: Qwen AI Research QA
45
+ - **Format**: GGUF (Q4_K_M Quantization)
46
+ - **Primary Use Case**: AI research question answering
47
+ - **Inference Framework**: `llama-cpp-python`
48
+ - **Optimized for**: Running on local hardware with reduced memory usage
49
+
50
+ ## License
51
+ This model is open-source and available under the **MIT License**.
52
+
53
+ ## Acknowledgments
54
+ This model is hosted by **InduwaraR** on Hugging Face. Special thanks to the **Qwen AI team** for their contributions to AI research and development.