runtime error

Exit code: 1. Reason: elf.__name__) File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1050, in _gcd_import File "<frozen importlib._bootstrap>", line 1027, in _find_and_load File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 688, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 883, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/usr/local/lib/python3.10/site-packages/transformers/models/qwen2_5_vl/modeling_qwen2_5_vl.py", line 39, in <module> from ...modeling_flash_attention_utils import flash_attn_supports_top_left_mask, is_flash_attn_available File "/usr/local/lib/python3.10/site-packages/transformers/modeling_flash_attention_utils.py", line 36, in <module> from flash_attn.bert_padding import index_first_axis, pad_input, unpad_input # noqa File "/usr/local/lib/python3.10/site-packages/flash_attn/__init__.py", line 3, in <module> from flash_attn.flash_attn_interface import ( File "/usr/local/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 15, in <module> import flash_attn_2_cuda as flash_attn_gpu ModuleNotFoundError: No module named 'flash_attn_2_cuda' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/user/app/app.py", line 17, in <module> from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor File "/usr/local/lib/python3.10/site-packages/transformers/utils/import_utils.py", line 2048, in __getattr__ raise ModuleNotFoundError( ModuleNotFoundError: Could not import module 'Qwen2_5_VLForConditionalGeneration'. Are this object's requirements defined correctly? /usr/local/lib/python3.10/site-packages/torch/cuda/__init__.py:789: UserWarning: Can't initialize NVML warnings.warn("Can't initialize NVML")

Container logs: