Smart_LLM

Running on Zero

Daemontatox commited on Jan 18

Commit

fc5ddd4

verified ·

1 Parent(s): 8e385dc

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -101,7 +101,7 @@ def initialize_model():
         device_map="cuda",
         attn_implementation="flash_attention_2",
         trust_remote_code=True,
-        #quantization_config=quantization_config
     )

         device_map="cuda",
         attn_implementation="flash_attention_2",
         trust_remote_code=True,
+        quantization_config=quantization_config
     )