Quantized models
Collection
Select models helpfully quantized by others as well as myself
•
59 items
•
Updated
•
2
EXL2 quants of meta-llama/Llama-3.2-1B-Instruct by branch:
- 4_0 : 4.0 bits per weight
- 5_0 : 5.0 bits per weight
- 6_0 : 6.0 bits per weight
- 8_0 : 8.0 bits per weight
Make your own EXL2 quants with measurement.json.
Quanted with exllamav2 v0.2.4.
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model's library.
Model tree for grimjim/meta-llama-Llama-3.2-1B-Instruct-exl2
Base model
meta-llama/Llama-3.2-1B-Instruct