How to select fairseq option `--ddp-backend`
I'm learning how to use fairseq to implement a simple translation model based on Transformer.
I would like to use 2 GeForce RTX 3090 GPUs on my lab server. Which option should I select for --ddp-backend of fairseq-train?
Furthermore, could you explain about the meaning of all following options for --ddp-backend and when to use them respectively?

From fairseq Documentation: Command-line Tools =&gt; fairseq-train =&gt; distributed_training

--ddp-backend:
Possible choices: c10d, fully_sharded, legacy_ddp, no_c10d, pytorch_ddp, slowmo
DistributedDataParallel backend
Default: “pytorch_ddp”

I'm new to stack exchange community, sorry if there is any inappropriate action.

I am not too sure, but I found this on GitHub
DDP_BACKEND_CHOICES = ChoiceEnum(
    [
        &quot;c10d&quot;,  # alias for pytorch_ddp
        &quot;fully_sharded&quot;,  # FullyShardedDataParallel from fairscale
        &quot;legacy_ddp&quot;,
        &quot;no_c10d&quot;,  # alias for legacy_ddp
        &quot;pytorch_ddp&quot;,
        &quot;slowmo&quot;,
    ]
)

Might be helpful, but I am also struggling with this
