Hugging Face's logo

tensorblock
/
starcoder2-15b-instruct-v0.1-GGUF

Text Generation
Transformers
GGUF
code
TensorBlock
GGUF
Eval Results
conversational
Model card Files Community
morriszms commited on
Commit
7adf391
·
verified ·
1 Parent(s): 537d8ea

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -12
README.md CHANGED
@@ -103,8 +103,16 @@ This repo contains GGUF format model files for [bigcode/starcoder2-15b-instruct-
103
 
104
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
105
 
 
 
 
 
 
 
 
106
  ## Prompt template
107
 
 
108
  ```
109
  <|endoftext|>You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.
110
 
@@ -118,18 +126,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
118
 
119
  | Filename | Quant type | File Size | Description |
120
  | -------- | ---------- | --------- | ----------- |
121
- | [starcoder2-15b-instruct-v0.1-Q2_K.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q2_K.gguf) | Q2_K | 5.768 GB | smallest, significant quality loss - not recommended for most purposes |
122
- | [starcoder2-15b-instruct-v0.1-Q3_K_S.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q3_K_S.gguf) | Q3_K_S | 6.507 GB | very small, high quality loss |
123
- | [starcoder2-15b-instruct-v0.1-Q3_K_M.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q3_K_M.gguf) | Q3_K_M | 7.492 GB | very small, high quality loss |
124
- | [starcoder2-15b-instruct-v0.1-Q3_K_L.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q3_K_L.gguf) | Q3_K_L | 8.350 GB | small, substantial quality loss |
125
- | [starcoder2-15b-instruct-v0.1-Q4_0.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q4_0.gguf) | Q4_0 | 8.443 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
126
- | [starcoder2-15b-instruct-v0.1-Q4_K_S.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q4_K_S.gguf) | Q4_K_S | 8.532 GB | small, greater quality loss |
127
- | [starcoder2-15b-instruct-v0.1-Q4_K_M.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q4_K_M.gguf) | Q4_K_M | 9.183 GB | medium, balanced quality - recommended |
128
- | [starcoder2-15b-instruct-v0.1-Q5_0.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q5_0.gguf) | Q5_0 | 10.265 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
129
- | [starcoder2-15b-instruct-v0.1-Q5_K_S.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q5_K_S.gguf) | Q5_K_S | 10.265 GB | large, low quality loss - recommended |
130
- | [starcoder2-15b-instruct-v0.1-Q5_K_M.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q5_K_M.gguf) | Q5_K_M | 10.646 GB | large, very low quality loss - recommended |
131
- | [starcoder2-15b-instruct-v0.1-Q6_K.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q6_K.gguf) | Q6_K | 12.201 GB | very large, extremely low quality loss |
132
- | [starcoder2-15b-instruct-v0.1-Q8_0.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/tree/main/starcoder2-15b-instruct-v0.1-Q8_0.gguf) | Q8_0 | 15.800 GB | very large, extremely low quality loss - not recommended |
133
 
134
 
135
  ## Downloading instruction
 
103
 
104
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
105
 
106
+
107
+ <div style="text-align: left; margin: 20px 0;">
108
+ <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
109
+ Run them on the TensorBlock client using your local machine ↗
110
+ </a>
111
+ </div>
112
+
113
  ## Prompt template
114
 
115
+
116
  ```
117
  <|endoftext|>You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.
118
 
 
126
 
127
  | Filename | Quant type | File Size | Description |
128
  | -------- | ---------- | --------- | ----------- |
129
+ | [starcoder2-15b-instruct-v0.1-Q2_K.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q2_K.gguf) | Q2_K | 5.768 GB | smallest, significant quality loss - not recommended for most purposes |
130
+ | [starcoder2-15b-instruct-v0.1-Q3_K_S.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_S.gguf) | Q3_K_S | 6.507 GB | very small, high quality loss |
131
+ | [starcoder2-15b-instruct-v0.1-Q3_K_M.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_M.gguf) | Q3_K_M | 7.492 GB | very small, high quality loss |
132
+ | [starcoder2-15b-instruct-v0.1-Q3_K_L.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q3_K_L.gguf) | Q3_K_L | 8.350 GB | small, substantial quality loss |
133
+ | [starcoder2-15b-instruct-v0.1-Q4_0.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_0.gguf) | Q4_0 | 8.443 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
134
+ | [starcoder2-15b-instruct-v0.1-Q4_K_S.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_K_S.gguf) | Q4_K_S | 8.532 GB | small, greater quality loss |
135
+ | [starcoder2-15b-instruct-v0.1-Q4_K_M.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q4_K_M.gguf) | Q4_K_M | 9.183 GB | medium, balanced quality - recommended |
136
+ | [starcoder2-15b-instruct-v0.1-Q5_0.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_0.gguf) | Q5_0 | 10.265 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
137
+ | [starcoder2-15b-instruct-v0.1-Q5_K_S.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_K_S.gguf) | Q5_K_S | 10.265 GB | large, low quality loss - recommended |
138
+ | [starcoder2-15b-instruct-v0.1-Q5_K_M.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q5_K_M.gguf) | Q5_K_M | 10.646 GB | large, very low quality loss - recommended |
139
+ | [starcoder2-15b-instruct-v0.1-Q6_K.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q6_K.gguf) | Q6_K | 12.201 GB | very large, extremely low quality loss |
140
+ | [starcoder2-15b-instruct-v0.1-Q8_0.gguf](https://huggingface.co/tensorblock/starcoder2-15b-instruct-v0.1-GGUF/blob/main/starcoder2-15b-instruct-v0.1-Q8_0.gguf) | Q8_0 | 15.800 GB | very large, extremely low quality loss - not recommended |
141
 
142
 
143
  ## Downloading instruction