title: Pytorch: why logging fails in DDP?
body: <p>I would like to use logging in one of the processes managed by Distributed Data Parallel. However, logging print nothing in the following codes (the codes are derived from <a href="https://pytorch.org/tutorials/intermediate/ddp_tutorial.html" rel="nofollow noreferrer">this tutorial</a>):</p>
  <pre><code>#!/usr/bin/python
  
  import os, logging
  # logging.basicConfig(level=logging.DEBUG)
  
  import torch
  
  def setup(rank, world_size):
      os.environ['MASTER_ADDR'] = 'localhost'
      os.environ['MASTER_PORT'] = '12355'
  
      # Initialize the process group.
      dist.init_process_group('NCCL', rank=rank, world_size=world_size)
  
  
  def cleanup():
      dist.destroy_process_group()
  
  
  class ToyModel(nn.Module):
      def __init__(self):
          super(ToyModel, self).__init__()
          self.net1 = nn.Linear(10, 10)
          self.relu = nn.ReLU()
          self.net2 = nn.Linear(10, 5)
  
      def forward(self, x):
          return self.net2(self.relu(self.net1(x)))
  
  
  def demo_basic(rank, world_size):
      setup(rank, world_size)
  
      if rank == 0:
          logger = logging.getLogger('train')
          logger.setLevel(logging.DEBUG)
          logger.info(f'Running DPP on rank={rank}.')
  
      # Create model and move it to GPU.
      model = ToyModel().to(rank)
      ddp_model = DDP(model, device_ids=[rank])
  
      loss_fn = nn.MSELoss()
      optimizer = optim.SGD(ddp_model.parameters(), lr=0.001)  # optimizer takes DDP model.
  
      optimizer.zero_grad()
      inputs = torch.randn(20, 10)  # .to(rank)
  
      outputs = ddp_model(inputs)
  
      labels = torch.randn(20, 5).to(rank)
      loss_fn(outputs, labels).backward()
  
      optimizer.step()
  
      cleanup()
  
  
  def run_demo(demo_func, world_size):
      mp.spawn(
          demo_func,
          args=(world_size,),
          nprocs=world_size,
          join=True
      )
  
  
  def main():
      run_demo(demo_basic, 4)
  
  
  if __name__ == &quot;__main__&quot;:
      main()
  </code></pre>
  <p>However, when I uncomment the 4th line, the logging works. May I know the reason and how to fix the bug please?</p>