Multi-Gpu Inference

by munikumar - opened Apr 15

Apr 15

•

How to run the inference with this model on multiple GPUs?
Can anyone please help me with this 🙏
@tahirjm

AI4Bharat org Jun 20

You will have to spawn multiple processes and load separate instance of this model on those GPUs (using .to('cuda:X')).

tahirjm changed discussion status to closed Jun 20

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment