communications to overlap with compute.
See this blog post for a more in-depth explanation and experimental
results, or read the docs and code at
torch/_dynamo/optimizations/distributed.py
To Debug DDPOptimizer, set torch._dynamo.config.log_level to DEBUG
(for full graph dumps) or INFO (for basic info about bucket
boundaries).  To disable DDPOptimizer, set
torch._dynamo.config.optimize_ddp=False. DDP and TorchDynamo should
still work correctly without DDPOptimizer, but with performance
degradation.