|
--- |
|
license: apache-2.0 |
|
language: en |
|
tags: |
|
- openbmb/MiniCPM4-0.5B |
|
- coding |
|
- code-generation |
|
- fine-tuned |
|
- qlora |
|
- gguf |
|
- instruction |
|
- python |
|
datasets: |
|
- TokenBender/code_instructions_122k_alpaca_style |
|
model_type: openbmb/MiniCPM4-0.5B |
|
base_model: openbmb/MiniCPM4-0.5B |
|
--- |
|
|
|
# MiniCPM4-0.5B-Coding-Finetuned-v1 |
|
|
|
This model is a fine-tuned version of `openbmb/MiniCPM4-0.5B` specialized for Python code generation tasks. It's designed to understand programming-related instructions and provide accurate and efficient Python code solutions. |
|
|
|
## π» Model Description |
|
|
|
- **Base Model**: `openbmb/MiniCPM4-0.5B` |
|
- **Fine-tuning Method**: **QLoRA** (Quantized Low-Rank Adaptation) |
|
- **Dataset**: `TokenBender/code_instructions_122k_alpaca_style` - A large dataset of coding instructions and their corresponding solutions. |
|
- **Training**: Optimized for instruction-based code generation using 4-bit quantization for efficiency. |
|
|
|
## β οΈ Important Considerations |
|
|
|
- **Verify All Code**: Generated code may contain errors or be suboptimal. Always test and review the code thoroughly before using it in production environments. |
|
- **Security**: The generated code has not been vetted for security vulnerabilities. Be cautious when using it in security-sensitive applications. |
|
- **Not a Replacement for Developers**: This model is a tool to assist developers, not replace them. Human oversight and expertise are crucial. |
|
|
|
## π Usage |
|
|
|
### With `transformers` |
|
|
|
```python |
|
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline |
|
import torch |
|
|
|
model_id = "rohitnagareddy/MiniCPM4-0.5B-Coding-Finetuned-v1" |
|
|
|
# Load model and tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) |
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_id, |
|
torch_dtype=torch.float16, |
|
device_map="auto", |
|
trust_remote_code=True |
|
) |
|
|
|
# Create conversation for a Python code-generation task |
|
messages = [ |
|
{"role": "system", "content": "You are an expert coding assistant."}, |
|
{"role": "user", "content": "Write a Python function that takes a list of integers and returns the sum of all even numbers in the list."} |
|
] |
|
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
|
|
|
pipe = pipeline( |
|
"text-generation", |
|
model=model, |
|
tokenizer=tokenizer |
|
) |
|
|
|
# Generate response |
|
outputs = pipe(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) |
|
print(outputs[0]["generated_text"]) |
|
``` |
|
|
|
## π§ GGUF Versions |
|
|
|
This repository includes quantized GGUF versions for use with `llama.cpp` and compatible tools: |
|
|
|
- `MiniCPM4-0.5B-Coding-Finetuned-v1.fp16.gguf` - Full precision (largest, best quality) |
|
- `MiniCPM4-0.5B-Coding-Finetuned-v1.Q8_0.gguf` - 8-bit quantization (good balance) |
|
- `MiniCPM4-0.5B-Coding-Finetuned-v1.Q5_K_M.gguf` - 5-bit quantization (smaller, fast) |
|
- `MiniCPM4-0.5B-Coding-Finetuned-v1.Q4_K_M.gguf` - 4-bit quantization (smallest, fastest) |
|
|
|
### Example with llama.cpp |
|
|
|
```bash |
|
./main -m ./MiniCPM4-0.5B-Coding-Finetuned-v1.Q4_K_M.gguf -n 256 -p "<|im_start|>system\nYou are an expert coding assistant.<|im_end|>\n<|im_start|>user\nCreate a Python function to find the factorial of a number.<|im_end|>\n<|im_start|>assistant\n" |
|
``` |
|
|
|
## π Training Details |
|
|
|
- **Training Epochs**: 1 |
|
- **QLoRA Rank (r)**: 16 |
|
- **QLoRA Alpha**: 32 |
|
- **Learning Rate**: 2e-4 |
|
- **Optimizer**: Paged AdamW 32-bit |
|
- **Target Modules**: Auto-detected linear layers |
|
|
|
|
|
|