imal. See this issue for more information.


Model offloading for fast inference and memory savings
	
Sequential CPU offloading, as discussed in the previous section, preserves a lot of memory but makes inference slower, because submodules are moved to GPU as needed, and immediately returned to CPU when a new module runs.
Full-model offloading is an alternative that moves whole models to the GPU, instead of handling each model’s constituent mod