Spaces:

Aneeth
/

zephyr_infererence

Sleeping

Aneeth commited on Jan 20, 2024

Commit

733133f

verified ·

1 Parent(s): 45bb491

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -17,7 +17,7 @@ model = AutoPeftModelForCausalLM.from_pretrained(
     "Aneeth/zephyr_10k",
     low_cpu_mem_usage=True,
     return_dict=True,
-    torch_dtype=torch.float16,
     device_map="cpu",
     trust_remote_code=True,
     quantization_config = gptq_config

     "Aneeth/zephyr_10k",
     low_cpu_mem_usage=True,
     return_dict=True,
+    torch_dtype=torch.float32,
     device_map="cpu",
     trust_remote_code=True,
     quantization_config = gptq_config