This model is a quantized version of Sidrap-V2

Install llama.cpp
Download the model
~/llama.cpp/main -ngl 32 -m sidrap-7b-v2.q4_K_M.gguf --color -c 4096 --temp 0.7 --repeat_penalty 1.1 -n -1 -i -ins

GGUF

Model size

7.24B params

Architecture

llama

4-bit

8-bit

Inference Providers NEW

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The model has no library tag.