Spaces:

baconnier
/

old_napoleon

Runtime error

mmcgovern574 commited on Feb 8

Commit

3c0f01f

verified ·

1 Parent(s): 4487681

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -85,7 +85,7 @@ model = AutoModelForCausalLM.from_pretrained(
     MODEL_ID,
     device_map="auto",
     quantization_config=quantization_config,
-    use_flash_attention_2=True,  # Enable Flash Attention 2 for better performance
     torch_dtype=torch.bfloat16
 )

     MODEL_ID,
     device_map="auto",
     quantization_config=quantization_config,
+    attn_implementation="flash_attention_2",  # Updated Flash Attention 2 parameter
     torch_dtype=torch.bfloat16
 )