How to use fairseq interactive with multiple gpu?
I am trying to generate new prediction for the model, but I found it is not that intuitive to use fairseq. I found fairseq-interactive could help to generate with a good settings of batch_size, however, it seems that it will use 1 GPU at a time, I wonder if it is possible to use multiple GPU? Hope someone can kindly help!
Many thanks :)

You cannot do this natively within fairseq. The best way to do this is to shard your data and run fairseq-interactive on each shard in the background. Be sure to set CUDA_VISIBLE_DEVICES for each shard so you put each shard's generation on a different GPU. This advice also applies to fairseq-generate (which will be significantly faster for large inference jobs).
