Spaces:

Cylanoid
/

llama_4_Medical_Fraud_Detection

Paused

Cylanoid commited on Apr 22

Commit

a7aeb40

1 Parent(s): e73698a

Fix max_memory keys (use integer 0) or drop max_memory

Files changed (1) hide show

app.py CHANGED Viewed

@@ -46,7 +46,7 @@ model = Llama4ForConditionalGeneration.from_pretrained(
     torch_dtype=torch.bfloat16,
     device_map="auto",
     max_memory={                               # cap GPU usage to ~11 GiB
-        "0": "11GiB",
         "cpu": "200GiB"
     },
     quantization_config=quant_config,

     torch_dtype=torch.bfloat16,
     device_map="auto",
     max_memory={                               # cap GPU usage to ~11 GiB
+        0: "11GiB",
         "cpu": "200GiB"
     },
     quantization_config=quant_config,