File size: 3,377 Bytes
a5df84b 55030c4 6c74f9c 55030c4 a53d237 55030c4 a5df84b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 |
---
base_model: arcee-ai/Arcee-Maestro-7B-Preview
library_name: transformers
license: apache-2.0
tags:
- llama-cpp
- gguf-my-repo
---
# Triangle104/Arcee-Maestro-7B-Preview-Q4_K_S-GGUF
This model was converted to GGUF format from [`arcee-ai/Arcee-Maestro-7B-Preview`](https://huggingface.co/arcee-ai/Arcee-Maestro-7B-Preview) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
Refer to the [original model card](https://huggingface.co/arcee-ai/Arcee-Maestro-7B-Preview) for more details on the model.
---
Arcee-Maestro-7B-Preview (7B) is Arcee's first reasoning model trained with reinforment learning. It is based on the Qwen2.5-7B DeepSeek-R1 distillation DeepSeek-R1-Distill-Qwen-7B with further GPRO training. Though this is just a preview of our
upcoming work, it already shows promising improvements to mathematical
and coding abilities across a range of tasks.
Intended Use Cases
-
Advanced reasoning
Mathematics
Coding
Training & Fine-Tuning
-
Initial Training: Began with DeepSeek-R1-Distill-Qwen-7B
GRPO:
Trained on 450,000 verified math problems
Additional bootstrapped coding examples
Performance
-
Arcee-Maestro-7B-Preview shows strong performance in mathematics as
well as coding, competing against even O1 preview, a model far
surprassing its size.
Limitations
-
Context Length: 128k Tokens (may vary depending on the final tokenizer settings and system resources).
Knowledge Cut-off: Training data may not reflect the latest events or developments beyond June 2024.
Ethical Considerations
-
Content Generation Risks: Like any language model,
Arcee-Maestro-7B-Preview can generate potentially harmful or biased
content if prompted in certain ways.
License
-
Arcee-Maestro-7B-Preview (7B) is released under the Apache-2.0 License.
You are free to use, modify, and distribute this model in both
commercial and non-commercial applications, subject to the terms and
conditions of the license.
---
## Use with llama.cpp
Install llama.cpp through brew (works on Mac and Linux)
```bash
brew install llama.cpp
```
Invoke the llama.cpp server or the CLI.
### CLI:
```bash
llama-cli --hf-repo Triangle104/Arcee-Maestro-7B-Preview-Q4_K_S-GGUF --hf-file arcee-maestro-7b-preview-q4_k_s.gguf -p "The meaning to life and the universe is"
```
### Server:
```bash
llama-server --hf-repo Triangle104/Arcee-Maestro-7B-Preview-Q4_K_S-GGUF --hf-file arcee-maestro-7b-preview-q4_k_s.gguf -c 2048
```
Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
Step 1: Clone llama.cpp from GitHub.
```
git clone https://github.com/ggerganov/llama.cpp
```
Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
```
cd llama.cpp && LLAMA_CURL=1 make
```
Step 3: Run inference through the main binary.
```
./llama-cli --hf-repo Triangle104/Arcee-Maestro-7B-Preview-Q4_K_S-GGUF --hf-file arcee-maestro-7b-preview-q4_k_s.gguf -p "The meaning to life and the universe is"
```
or
```
./llama-server --hf-repo Triangle104/Arcee-Maestro-7B-Preview-Q4_K_S-GGUF --hf-file arcee-maestro-7b-preview-q4_k_s.gguf -c 2048
```
|