Running on Apple Silicon (Mac Mini M4 Pro)
When i try to build the docker image on my Mac, I get:Error response from daemon: unknown or invalid runtime name: nvidia.
the docker-compose file does have an item: runtime: nvidia.
Can this be changed? what would i use as runtime to run this on Apple Silicon?
Thank you
same problem here. it requires mamba-ssm which needs nvidia, also the code mentioned nvidia a few times..
Yeah, even I would request to make it available for apple silicon
same with me also, failed to run in Apple Silicon
I don't think there's metal inference available yet
From the GitHub repo:
"At the moment this repository only supports Linux systems (preferably Ubuntu 22.04/24.04) with recent NVIDIA GPUs (3000-series or newer, 6GB+ VRAM)."
Give it a bit of time, this is a very new project.
Mamba-ssm for Mac here https://github.com/alxndrTL/mamba.py/tree/main/mlx