|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
metrics: |
|
- accuracy |
|
base_model: |
|
- meta-llama/Llama-3.1-8B-Instruct |
|
pipeline_tag: reinforcement-learning |
|
--- |
|
# Hibernates-2B-R1-V1 |
|
|
|
A highly efficient 2B parameter language model optimized for reasoning and dialogue tasks. |
|
|
|
## Model Overview |
|
|
|
Hibernates-2B is a custom transformer architecture designed for advanced language understanding and generation. Built with performance and efficiency in mind, it leverages state-of-the-art techniques for natural language processing. |
|
|
|
### Key Features |
|
- 2B Parameters |
|
- 4096 Token Context Window |
|
- Custom Transformer Architecture |
|
- Optimized for CPU and GPU Inference |
|
- Multi-Turn Dialogue Support |
|
|
|
## Technical Specifications |
|
|
|
- **Architecture**: Custom Transformer |
|
- **Parameters**: 2 Billion |
|
- **Context Length**: 4096 tokens |
|
- **Model Type**: Decoder-only |
|
- **Tokenizer**: Custom WordPiece |
|
- **Format**: SafeTensors |
|
|
|
## Usage Guide |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
import torch |
|
|
|
# Load model and tokenizer |
|
model_id = "Hibernates-2B-R1-V1" |
|
tokenizer = AutoTokenizer.from_pretrained(model_id) |
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_id, |
|
torch_dtype=torch.float16, |
|
device_map="auto" |
|
) |
|
|
|
# Example conversation |
|
messages = [ |
|
{"role": "system", "content": "You are a helpful AI assistant."}, |
|
{"role": "user", "content": "How can you help me today?"} |
|
] |
|
|
|
# Generate response |
|
input_text = tokenizer.apply_chat_template(messages, tokenize=False) |
|
inputs = tokenizer(input_text, return_tensors="pt").to(model.device) |
|
outputs = model.generate( |
|
inputs["input_ids"], |
|
max_new_tokens=512, |
|
temperature=0.7, |
|
top_p=0.95 |
|
) |
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
``` |
|
|
|
## Performance Characteristics |
|
|
|
### Strengths |
|
- Efficient Resource Usage |
|
- Strong Reasoning Capabilities |
|
- Multi-Turn Dialogue |
|
- Context Awareness |
|
- Instruction Following |
|
|
|
### Considerations |
|
- Resource Requirements: 8GB+ GPU RAM recommended |
|
- Task Specificity: Best suited for dialogue and reasoning tasks |
|
- Language Support: Primary focus on English |
|
- Model Size: Optimized for balance of performance and efficiency |
|
|
|
## License and Usage |
|
- Research and commercial use permitted |
|
- Attribution appreciated but not required |
|
- No warranty provided |
|
|
|
## Citation |
|
|
|
If you use this model in your research, please cite: |
|
|
|
```bibtex |
|
@software{hibernates2b_2024, |
|
title={Hibernates-2B: Efficient Language Model for Reasoning}, |
|
year={2024}, |
|
version={R1-V1} |
|
} |
|
``` |
|
|
|
## Acknowledgments |
|
Built using PyTorch and Hugging Face Transformers. Special thanks to the open-source AI community. |
|
|
|
## Download Instructions |
|
|
|
Due to file size limitations, the model files are hosted externally. Download them from: |
|
|
|
1. [model-00001-of-00002.safetensors](https://huggingface.co/HibernatesAI/Hibernates-2B-R1-V1/blob/main/model-00001-of-00002.safetensors) |
|
2. [model-00002-of-00002.safetensors](https://huggingface.co/HibernatesAI/Hibernates-2B-R1-V1/blob/main/model-00002-of-00002.safetensors) |
|
|
|
Place these files in the root directory of the project before running. |
