Update quantization process
Browse files
README.md
CHANGED
|
@@ -107,7 +107,7 @@ To test the model against Hugging Face, you can use the following command:
|
|
| 107 |
|
| 108 |
```sh
|
| 109 |
# Example command for testing against Hugging Face
|
| 110 |
-
|
| 111 |
|
| 112 |
./main -m ./Refact-1_6B-fim/ggml-model-f16.gguf -n 300 -p "write a function to multiply two integers in python" --temp 1.0 --top-p 1.0 --top-k 1 --repeat_penalty 1.0
|
| 113 |
```
|
|
|
|
| 107 |
|
| 108 |
```sh
|
| 109 |
# Example command for testing against Hugging Face
|
| 110 |
+
python convert-hf-to-gguf-update.py ./Refact-1_6B-fim 1
|
| 111 |
|
| 112 |
./main -m ./Refact-1_6B-fim/ggml-model-f16.gguf -n 300 -p "write a function to multiply two integers in python" --temp 1.0 --top-p 1.0 --top-k 1 --repeat_penalty 1.0
|
| 113 |
```
|