Create README.md

README.md

@@ -0,0 +1,85 @@
+---
+language:
+- en
+base_model:
+- deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
+tags:
+- Qwen2
+- DeepSeek
+---
+# DeepSeek-R1-Distill-Qwen-7B Quantized Models
+
+This repository contains Q4_KM and Q5_KM quantized versions of the [DeepSeek-R1-Distill-Qwen-7B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) model, optimized for efficient deployment while maintaining strong performance.
+
+Discover our full range of quantized language models by visiting our [SandLogic Lexicon HuggingFace](https://huggingface.co/SandLogicTechnologies). To learn more about our company and services, check out our website at [SandLogic](https://www.sandlogic.com/).
+
+## Model Description
+
+These models are quantized versions of DeepSeek-R1-Distill-Qwen-7B, which is a distilled 7B parameter model based on the Qwen architecture. The model demonstrates that reasoning patterns from larger models can be effectively distilled into smaller architectures, resulting in exceptional performance on various benchmarks.
+
+### Key Features
+- Fine-tuned using DeepSeek-R1 generated reasoning data
+- Modified configurations and tokenizer optimized for performance
+- Maintains strong reasoning capabilities while reducing model size
+- Suitable for research and production deployment
+
+### Available Quantized Versions
+
+1. **Q4_KM Version**
+   - 4-bit quantization using the K-means method
+   - Approximately 4 GB model size
+   - Optimal balance between model size and performance
+   - Recommended for resource-constrained environments
+
+2. **Q5_KM Version**
+   - 5-bit quantization using the K-means method
+   - Approximately 4.5GB model size
+   - Higher precision than Q4 while maintaining significant size reduction
+   - Recommended when higher accuracy is needed
+
+## Usage
+
+```bash
+pip install llama-cpp-python 
+```
+Please refer to the llama-cpp-python [documentation](https://llama-cpp-python.readthedocs.io/en/latest/) to install with GPU support.
+
+### Basic Text Completion
+Here's an example demonstrating how to use the high-level API for basic text completion:
+
+```python
+from llama_cpp import Llama
+
+llm = Llama(
+    model_path="model/path/",
+    verbose=False,
+    # n_gpu_layers=-1, # Uncomment to use GPU acceleration
+    # n_ctx=2048, # Uncomment to increase the context window
+)
+
+# Example of a reasoning task
+output = llm(
+    "Q: Explain the concept of natural selection in simple terms. A: ",
+    max_tokens=256,
+    stop=["Q:", "\n\n"],
+    echo=False
+)
+
+print(output["choices"][0]["text"])
+```
+
+
+
+## Model Configuration Changes
+
+Please note that DeepSeek have made slight modifications to the original Qwen-7B configurations and tokenizer to optimize performance. When using these models, ensure you're using  provided settings rather than the original Qwen-7B configurations.
+
+## License
+
+This model inherits the license of the original DeepSeek-R1-Distill-Qwen-7B model. Please refer to the original model's license for usage terms and conditions.
+
+
+
+## Acknowledgments
+
+We thank the DeepSeek AI team for open-sourcing their distilled models and demonstrating that smaller models can achieve impressive performance through effective distillation techniques. Special thanks also to the Qwen team for providing the base model architecture.