inference can support multi gpu？

#11

by loong - opened Jul 13, 2024

loong

Jul 13, 2024

inference can support multi gpu？

Jul 13, 2024

@loong yes should be possible by setting device_map='balanced'

burkaygur changed discussion status to closed Jul 18, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment