Patryk Ptasiński Claude commited on
Commit
13e3d2b
·
1 Parent(s): 1a9d56e

Move xformers disable to global scope for better compatibility

Browse files

The previous fix didn't work because xformers was already imported.
Moving XFORMERS_DISABLED=1 to the top of the file before any imports
to ensure it's set before xformers library initialization.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>

Files changed (1) hide show
  1. app.py +4 -5
app.py CHANGED
@@ -1,6 +1,10 @@
1
  from typing import List, Dict, Any
2
  import json
3
  import torch
 
 
 
 
4
 
5
  import gradio as gr
6
  from fastapi import FastAPI
@@ -72,11 +76,6 @@ def load_model(model_name: str):
72
  try:
73
  print(f"Loading model '{model_name}' on {DEVICE}")
74
 
75
- # Special handling for Stella models on CPU
76
- if "stella" in model_name.lower() and DEVICE == "cpu":
77
- # Disable xformers for CPU inference to avoid attention operator errors
78
- import os
79
- os.environ["XFORMERS_DISABLED"] = "1"
80
 
81
  current_model = SentenceTransformer(
82
  model_name,
 
1
  from typing import List, Dict, Any
2
  import json
3
  import torch
4
+ import os
5
+
6
+ # Disable xformers for CPU compatibility with Stella models
7
+ os.environ["XFORMERS_DISABLED"] = "1"
8
 
9
  import gradio as gr
10
  from fastapi import FastAPI
 
76
  try:
77
  print(f"Loading model '{model_name}' on {DEVICE}")
78
 
 
 
 
 
 
79
 
80
  current_model = SentenceTransformer(
81
  model_name,