How to run Stable Diffusion with Core ML Core ML is the model format and machine learning library supported by Apple frameworks. If you are interested in running Stable Diffusion models inside your macOS or iOS/iPadOS apps, this guide will show you how to convert existing PyTorch checkpoints into the Core ML format and use them for inference with Python or Swift. Core ML models can leverage all the compute engines available in Apple devices: the CPU, the GPU, and the Apple Neural Engine (or ANE, a tensor-optimized accelerator available in Apple Silicon Macs and modern iPhones/iPads). Depending on the model and the device it’s running on, Core ML can mix and match compute engines too, so some portions of the model may run on the CPU while others run on GPU, for example. You can also run the diffusers Python codebase on Apple Silicon Macs using the mps accelerator built into PyTorch. This approach is explained in depth in the mps guide, but it is not compatible with native apps.  Stable Diffusion Core ML Checkpoints Stable Diffusion weights (or checkpoints) are stored in the PyTorch format, so you need to convert them to the Core ML format before we can use them inside native apps. Thankfully, Apple engineers developed a conversion tool based on diffusers to convert the PyTorch checkpoints to Core ML. Before you convert a model, though, take a moment to explore the Hugging Face Hub – chances are the model you’re interested in is already available in Core ML format: the Apple organization includes Stable Diffusion versions 1.4, 1.5, 2.0 base, and 2.1 base coreml community includes custom finetuned models use this filter to return all available Core ML checkpoints If you can’t find the model you’re interested in, we recommend you follow the instructions for Converting Models to Core ML by Apple.  Selecting the Core ML Variant to Use Stable Diffusion models can be converted to different Core ML variants intended for different purposes: The type of attention blocks used. The attention operation is used to “pay attention” to the relationship between different areas in the image representations and to understand how the image and text representations are related. Attention is compute- and memory-intensive, so different implementations exist that consider the hardware characteristics of different devices. For Core ML Stable Diffusion models, there are two attention variants: split_einsum (introduced by Apple) is optimized for ANE devices, which is available in modern iPhones, iPads and M-series computers. The “original” attention (the base implementation used in diffusers) is only compatible with CPU/GPU and not ANE. It can be faster to run your model on CPU + GPU using original attention than ANE. See this performance benchmark as well as some additional measures provided by the community for additional details. The supported inference framework. packages are suitable for Python inference. This can be used to test converted Core ML models before attempting to integrate them inside native apps, or if you want to explore Core ML performance but don’t need to support native apps. For example, an application with a web UI could perfectly use a Python Core ML backend. compiled models are required for Swift code. The compiled models in the Hub split the large UNet model weights into several files for compatibility with iOS and iPadOS devices. This corresponds to the --chunk-unet conversion option. If you want to support native apps, then you need to select the compiled variant. The official Core ML Stable Diffusion models include these variants, but the community ones may vary:   Copied coreml-stable-diffusion-v1-4