equires full float32 precision we recommend enabling this setting for matrix multiplications, too. It can significantly speed up computations with typically negligible loss of numerical accuracy. You can read more about it here. All you need to do is to add this before your inference:


	Copied
import torch

torch.backends.cuda.matmul.allow_tf32 = True

Half precision weights
	
To save more GPU memory and get more speed, you can load and run th