image = pipe(prompt).images[0] CPU offloading works on submodules rather than whole models. This is the best way to minimize memory consumption, but inference is much slower due to the iterative nature of the diffusion process. The UNet component of the pipeline runs several times (as many as num_inference_steps); each time, the different UNet submodules are sequentially onloaded and offloaded as needed, resulting in a large number of memory transfers. Consider using model offloading if you want to optimize for speed because it is much faster. The tradeoff is your memory savings won’t be as large. When using enable_sequential_cpu_offload(), don’t move the pipeline to CUDA beforehand or else the gain in memory consumption will only be minimal (see this issue for more information). enable_sequential_cpu_offload() is a stateful operation that installs hooks on the models.  Model offloading Model offloading requires 🤗 Accelerate version 0.17.0 or higher. Sequential CPU offloading preserves a lot of memory but it makes inference slower because submodules are moved to GPU as needed, and they’re immediately returned to the CPU when a new module runs. Full-model offloading is an alternative that moves whole models to the GPU, instead of handling each model’s constituent submodules. There is a negligible impact on inference time (compared with moving the pipeline to cuda), and it still provides some memory savings. During model offloading, only one of the main components of the pipeline (typically the text encoder, UNet and VAE)