Go to llama.cpp and download one of those folders
If you're about to use CUDA - check the version your card supports(12.2 for any RTX) and download one of those folders
Unpack everything in one folder and rename it to "LlamaCPP", put this folder in the same folder where main.py/main.exe file is
Launch main.py/main.exe file
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.