Update quantization process
Browse files
README.md
CHANGED
@@ -107,7 +107,7 @@ To test the model against Hugging Face, you can use the following command:
|
|
107 |
|
108 |
```sh
|
109 |
# Example command for testing against Hugging Face
|
110 |
-
|
111 |
|
112 |
./main -m ./Refact-1_6B-fim/ggml-model-f16.gguf -n 300 -p "write a function to multiply two integers in python" --temp 1.0 --top-p 1.0 --top-k 1 --repeat_penalty 1.0
|
113 |
```
|
|
|
107 |
|
108 |
```sh
|
109 |
# Example command for testing against Hugging Face
|
110 |
+
python convert-hf-to-gguf-update.py ./Refact-1_6B-fim 1
|
111 |
|
112 |
./main -m ./Refact-1_6B-fim/ggml-model-f16.gguf -n 300 -p "write a function to multiply two integers in python" --temp 1.0 --top-p 1.0 --top-k 1 --repeat_penalty 1.0
|
113 |
```
|