tch_size=4,
)
For more information, check out Optimum Habana’s documentation and the example provided in the official Github repository.

Benchmark
	
Here are the latencies for Habana Gaudi 1 and Gaudi 2 with the Habana/stable-diffusion Gaudi configuration (mixed precision bf16/fp32):

Latency
Batch size
Gaudi 1
4.37s
4/8
Gaudi 2
1.19s
4/8
