ns.
Full-model offloading is an alternative that moves whole models to the GPU, instead of handling each model’s constituent modules. This results in a negligible impact on inference time (compared with moving the pipeline to cuda), while still providing some memory savings.
In this scenario, only one of the main components of the pipeline (typically: text encoder, unet and vae)
will be in the GPU while the others wait in the CPU. Components li