Spaces:

simone-papicchio
/

qatch-demo

Sleeping

simone-papicchio commited on Apr 6

Commit

7e9dee7

1 Parent(s): ccba587

fix: torch not defined

Files changed (1) hide show

prediction.py CHANGED Viewed

@@ -27,7 +27,7 @@ from tqdm import tqdm
 pipeline = hf_pipeline(
     "text-generation",
     model="meta-llama/Meta-Llama-3.1-8B-Instruct",
-    model_kwargs={"torch_dtype": torch.bfloat16},
 )
 pipeline.to('cuda')

 pipeline = hf_pipeline(
     "text-generation",
     model="meta-llama/Meta-Llama-3.1-8B-Instruct",
+    model_kwargs={"torch_dtype": 'bfloat16'},
 )
 pipeline.to('cuda')