Patryk Ptasiński Claude commited on
Commit
1a9d56e
·
1 Parent(s): 3726350

Fix Stella models CPU compatibility by disabling xformers

Browse files

Stella models were failing on CPU with xformers attention operator errors.
Added environment variable XFORMERS_DISABLED=1 for Stella models on CPU
to force fallback to standard PyTorch attention mechanisms.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

Files changed (1) hide show
  1. app.py +7 -0
app.py CHANGED
@@ -71,6 +71,13 @@ def load_model(model_name: str):
71
  trust_remote_code = MODELS.get(model_name, {}).get("trust_remote_code", False)
72
  try:
73
  print(f"Loading model '{model_name}' on {DEVICE}")
 
 
 
 
 
 
 
74
  current_model = SentenceTransformer(
75
  model_name,
76
  trust_remote_code=trust_remote_code,
 
71
  trust_remote_code = MODELS.get(model_name, {}).get("trust_remote_code", False)
72
  try:
73
  print(f"Loading model '{model_name}' on {DEVICE}")
74
+
75
+ # Special handling for Stella models on CPU
76
+ if "stella" in model_name.lower() and DEVICE == "cpu":
77
+ # Disable xformers for CPU inference to avoid attention operator errors
78
+ import os
79
+ os.environ["XFORMERS_DISABLED"] = "1"
80
+
81
  current_model = SentenceTransformer(
82
  model_name,
83
  trust_remote_code=trust_remote_code,