Update README.md

README.md

@@ -122,6 +122,69 @@ Under Download Model, you can enter the model repo: infosys/NT-Java-1.1B-GGUF an
 
 Then click Download.
 
+### On the command line, including multiple files at once
+
+I recommend using the `huggingface-hub` Python library:
+
+```shell
+pip3 install huggingface-hub
+```
+
+Then you can download any individual model file to the current directory, at high speed, with a command like this:
+
+```shell
+huggingface-cli download infosys/NT-Java-1.1B-GGUF NT-Java-1.1B_Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
+```
+
+<details>
+  <summary>More advanced huggingface-cli download usage (click to read)</summary>
+
+You can also download multiple files at once with a pattern:
+
+```shell
+huggingface-cli download infosys/NT-Java-1.1B-GGUF --local-dir . --local-dir-use-symlinks False --include='*Q4_K*gguf'
+```
+
+For more documentation on downloading with `huggingface-cli`, please see: [HF -> Hub Python Library -> Download files -> Download from the CLI](https://huggingface.co/docs/huggingface_hub/guides/download#download-from-the-cli).
+
+To accelerate downloads on fast connections (1Gbit/s or higher), install `hf_transfer`:
+
+```shell
+pip3 install hf_transfer
+```
+
+And set environment variable `HF_HUB_ENABLE_HF_TRANSFER` to `1`:
+
+```shell
+HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download infosys/NT-Java-1.1B-GGUF NT-Java-1.1B_Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
+```
+
+Windows Command Line users: You can set the environment variable by running `set HF_HUB_ENABLE_HF_TRANSFER=1` before the download command.
+</details>
+<!-- README_GGUF.md-how-to-download end -->
+
+<!-- README_GGUF.md-how-to-run start -->
+## Example `llama.cpp` command
+
+Make sure you are using `llama.cpp` from commit [d0cee0d](https://github.com/ggerganov/llama.cpp/commit/d0cee0d36d5be95a0d9088b674dbb27354107221) or later.
+
+```shell
+./main -ngl 35 -m NT-Java-1.1B_Q4_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "[INST] {prompt} [/INST]"
+```
+
+Change `-ngl 32` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
+
+Change `-c 2048` to the desired sequence length. For extended sequence models - eg 8K, 16K, 32K - the necessary RoPE scaling parameters are read from the GGUF file and set by llama.cpp automatically. Note that longer sequence lengths require much more resources, so you may need to reduce this value.
+
+If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`
+
+For other parameters and how to use them, please refer to [the llama.cpp documentation](https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md)
+
+## How to run in `text-generation-webui`
+
+Further instructions here: [text-generation-webui/docs/04 - Model Tab.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/04%20-%20Model%20Tab.md#llamacpp).
+
+
 
 ## How to use with Ollama
 
@@ -193,68 +256,6 @@ public class HelloWorld {\n    public static void main(String[] args) {
 Your browser should open automatically and display a chat interface. (If it doesn't, just open your browser and point it at http://localhost:8080)
 
 
-### On the command line, including multiple files at once
-
-I recommend using the `huggingface-hub` Python library:
-
-```shell
-pip3 install huggingface-hub
-```
-
-Then you can download any individual model file to the current directory, at high speed, with a command like this:
-
-```shell
-huggingface-cli download infosys/NT-Java-1.1B-GGUF NT-Java-1.1B_Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
-```
-
-<details>
-  <summary>More advanced huggingface-cli download usage (click to read)</summary>
-
-You can also download multiple files at once with a pattern:
-
-```shell
-huggingface-cli download infosys/NT-Java-1.1B-GGUF --local-dir . --local-dir-use-symlinks False --include='*Q4_K*gguf'
-```
-
-For more documentation on downloading with `huggingface-cli`, please see: [HF -> Hub Python Library -> Download files -> Download from the CLI](https://huggingface.co/docs/huggingface_hub/guides/download#download-from-the-cli).
-
-To accelerate downloads on fast connections (1Gbit/s or higher), install `hf_transfer`:
-
-```shell
-pip3 install hf_transfer
-```
-
-And set environment variable `HF_HUB_ENABLE_HF_TRANSFER` to `1`:
-
-```shell
-HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download infosys/NT-Java-1.1B-GGUF NT-Java-1.1B_Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
-```
-
-Windows Command Line users: You can set the environment variable by running `set HF_HUB_ENABLE_HF_TRANSFER=1` before the download command.
-</details>
-<!-- README_GGUF.md-how-to-download end -->
-
-<!-- README_GGUF.md-how-to-run start -->
-## Example `llama.cpp` command
-
-Make sure you are using `llama.cpp` from commit [d0cee0d](https://github.com/ggerganov/llama.cpp/commit/d0cee0d36d5be95a0d9088b674dbb27354107221) or later.
-
-```shell
-./main -ngl 35 -m NT-Java-1.1B_Q4_K_M.gguf --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -p "[INST] {prompt} [/INST]"
-```
-
-Change `-ngl 32` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
-
-Change `-c 2048` to the desired sequence length. For extended sequence models - eg 8K, 16K, 32K - the necessary RoPE scaling parameters are read from the GGUF file and set by llama.cpp automatically. Note that longer sequence lengths require much more resources, so you may need to reduce this value.
-
-If you want to have a chat-style conversation, replace the `-p <PROMPT>` argument with `-i -ins`
-
-For other parameters and how to use them, please refer to [the llama.cpp documentation](https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md)
-
-## How to run in `text-generation-webui`
-
-Further instructions here: [text-generation-webui/docs/04 - Model Tab.md](https://github.com/oobabooga/text-generation-webui/blob/main/docs/04%20-%20Model%20Tab.md#llamacpp).
-
 ## How to run from Python code
 
 You can use GGUF models from Python using the [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) version 0.2.23 and later.