tch_size=4,
)
For more information, check out Optimum Habana’s documentation and the example provided in the official Github repository.

Benchmark
	
Here are the latencies for Habana first-generation Gaudi and Gaudi2 with the Habana/stable-diffusion Gaudi configuration (mixed precision bf16/fp32):
Stable Diffusion v1.5 (512x512 resolution):

Latency (batch size = 1)
Throughput (batch size = 8)
first-generation Gaudi
4.22s
0.29 images/s
Gaudi2
