Spaces:
Running
Running
| title: README | |
| emoji: 😻 | |
| colorFrom: purple | |
| colorTo: yellow | |
| sdk: static | |
| pinned: false | |
|  | |
| Run Open source LLMs and create OpenAI-compatible API services for the Llama2 series of LLMs locally With LlamaEdge! | |
| ## Give it a try | |
| Run a single command in your command line terminal. | |
| ``` | |
| bash <(curl -sSfL 'https://raw.githubusercontent.com/LlamaEdge/LlamaEdge/main/run-llm.sh') --interactive | |
| ``` | |
| Follow the on-screen instructions to install the WasmEdge Runtime and download your favorite open-source LLM. Then, choose whether you want to chat with the model via the CLI or via a web UI. | |
| [See it in action](https://youtu.be/Hqu-PBqkzDk) | [GitHub](https://github.com/LlamaEdge/LlamaEdge) | [Docs](https://www.secondstate.io/articles/run-llm-sh/) | |
| ## Why? | |
| [LlamaEdge](https://github.com/second-state/LlamaEdge), powered by Rust and WasmEdge, provides a strong alternative to Python in AI inference. | |
| * Lightweight. The total runtime size is 30MB. | |
| * Fast. Full native speed on GPUs. | |
| * **Portable. Single cross-platform binary on different CPUs, GPUs, and OSes.** | |
| * Secure. Sandboxed and isolated execution on untrusted devices. | |
| * Container-ready. Supported in Docker, containerd, Podman, and Kubernetes. | |
| ## Learn more | |
| Please visit the [LlamaEdge](https://github.com/second-state/LlamaEdge) project to learn more. | |