Qwen2.5-0.5B-Rag-Thinking-Flan-T5

Running

Akjava commited on Mar 19

Commit

22ce07c

verified ·

1 Parent(s): 35ac2bb

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -215,7 +215,7 @@ def respond(
                 model_path=f"models/{model}",
                 flash_attn=False,
                 n_gpu_layers=0,
-                n_batch=8,
                 n_ctx=2048,
                 n_threads=2,
                 n_threads_batch=2,

                 model_path=f"models/{model}",
                 flash_attn=False,
                 n_gpu_layers=0,
+                n_batch=16,
                 n_ctx=2048,
                 n_threads=2,
                 n_threads_batch=2,