Update README.md
Browse files
README.md
CHANGED
@@ -80,49 +80,16 @@ CMAKE_ARGS="-DGGML_METAL=on" pip install --upgrade --force-reinstall llama-cpp-p
|
|
80 |
```
|
81 |
|
82 |
### Dependencies
|
83 |
-
Create a `requirements.txt` file with the following contents:
|
84 |
-
```text
|
85 |
-
black==24.10.0
|
86 |
-
certifi==2024.8.30
|
87 |
-
charset-normalizer==3.4.0
|
88 |
-
click==8.1.7
|
89 |
-
diskcache==5.6.3
|
90 |
-
filelock==3.16.1
|
91 |
-
fsspec==2024.10.0
|
92 |
-
huggingface-hub==0.26.2
|
93 |
-
idna==3.10
|
94 |
-
iniconfig==2.0.0
|
95 |
-
isort==5.13.2
|
96 |
-
Jinja2==3.1.4
|
97 |
-
llama_cpp_python==0.3.2
|
98 |
-
markdown-it-py==3.0.0
|
99 |
-
MarkupSafe==3.0.2
|
100 |
-
mdurl==0.1.2
|
101 |
-
mypy-extensions==1.0.0
|
102 |
-
numpy==2.1.3
|
103 |
-
packaging==24.2
|
104 |
-
pathspec==0.12.1
|
105 |
-
platformdirs==4.3.6
|
106 |
-
pluggy==1.5.0
|
107 |
-
psutil==6.1.0
|
108 |
-
Pygments==2.18.0
|
109 |
-
pytest==8.3.3
|
110 |
-
PyYAML==6.0.2
|
111 |
-
requests==2.32.3
|
112 |
-
rich==13.9.4
|
113 |
-
tqdm==4.67.0
|
114 |
-
typing_extensions==4.12.2
|
115 |
-
urllib3==2.2.3
|
116 |
-
```
|
117 |
|
118 |
-
Then install dependencies
|
119 |
```bash
|
120 |
-
pip install -r
|
121 |
```
|
122 |
|
123 |
## Inference Loop
|
124 |
|
125 |
```python
|
|
|
126 |
import os
|
127 |
import time
|
128 |
import logging
|
@@ -466,6 +433,10 @@ if __name__ == "__main__":
|
|
466 |
main()
|
467 |
```
|
468 |
|
|
|
|
|
|
|
|
|
469 |
### Nvidia Llama 3 - ChatQA Paper:
|
470 |
Arxiv : [https://arxiv.org/pdf/2401.10225](https://arxiv.org/pdf/2401.10225)
|
471 |
|
|
|
80 |
```
|
81 |
|
82 |
### Dependencies
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
83 |
|
84 |
+
Then install dependencies (inference_requirements.txt) attached under `Files and Versions`:
|
85 |
```bash
|
86 |
+
pip install -r inference_requirements.txt
|
87 |
```
|
88 |
|
89 |
## Inference Loop
|
90 |
|
91 |
```python
|
92 |
+
# Attached under `Files and Versions` (inference_open-insurance-llm-gguf.py)
|
93 |
import os
|
94 |
import time
|
95 |
import logging
|
|
|
433 |
main()
|
434 |
```
|
435 |
|
436 |
+
```bash
|
437 |
+
python3 inference_open-insurance-llm-gguf.py
|
438 |
+
```
|
439 |
+
|
440 |
### Nvidia Llama 3 - ChatQA Paper:
|
441 |
Arxiv : [https://arxiv.org/pdf/2401.10225](https://arxiv.org/pdf/2401.10225)
|
442 |
|