GAN, discriminator output only 0 or 1
I'm trying to train SRGAN. (Super Resolution GAN)
However, the discriminator's output converge to 0 or 1 whatever the input is.
Discriminator's loss function is only 

D_loss = 0.5*(D_net(fake) + 1 - D_net(real))


D_net(fake) and D_net(real) both becomes 0 or 1. (sigmoid)

How can I fix it?

    for epoch_idx in range(epoch_num):
      for batch_idx, data in enumerate(data_loader):
        D_net.zero_grad()
        #### make real, low, fake
        real = data[0]
        for img_idx in range(batch_size):
            low[img_idx] = trans_low_res(real[img_idx])
        fake = G_net(Variable(low).cuda())

        #### get Discriminator loss and train Discriminator
        real_D_out = D_net(Variable(real).cuda()).mean()
        fake_D_out = D_net(Variable(fake).cuda()).mean()

        D_loss = 0.5*(fake_D_out + 1 - real_D_out)
        D_loss.backward()
        D_optim.step()

        #### train Generator

        G_net.zero_grad()
        #### get new fake D out with updated Discriminator
        fake_D_out = D_net(Variable(fake).cuda()).mean()
        G_loss = generator_criterion(fake_D_out.cuda(), fake.cuda(), real.cuda())
        G_loss.backward()
        G_optim.step()



  Batch : [10/6700] Discriminator_Loss: 0.0860 Generator_Loss : 0.1393 
  
  Batch : [20/6700] Discriminator_Loss: 0.0037 Generator_Loss : 0.1282 
  
  Batch : [30/6700] Discriminator_Loss: 0.0009 Generator_Loss : 0.0838 
  
  Batch : [40/6700] Discriminator_Loss: 0.0002 Generator_Loss : 0.0735 
  
  Batch : [50/6700] Discriminator_Loss: 0.0001 Generator_Loss : 0.0648 
  
  Batch : [60/6700] Discriminator_Loss: 0.5000 Generator_Loss : 0.0634 
  
  Batch : [70/6700] Discriminator_Loss: 0.5000 Generator_Loss : 0.0706 
  
  Batch : [80/6700] Discriminator_Loss: 0.5000 Generator_Loss : 0.0691 
  
  Batch : [90/6700] Discriminator_Loss: 0.5000 Generator_Loss : 0.0538 
  ...


I am not sure if I understand your problem correctly. You meant that the sigmoid output from the discriminator is either 0 or 1? 

In your loss function: D_loss = 0.5 * (fake_D_out + 1 - real_D_out), you are directly optimizing on the sigmoid output and looks like the discriminator overfits to your data that it can accurately predict 0 and 1 for fake and real examples respectively.

There are some GAN hacks suggested by experts in this subject matters. You can find a list of tips and tricks here. I would like to suggest you use soft-labels rather than hard-labels (see ref).

You can use BCEWithLogitsLoss() and compute loss based on soft labels instead of hard labels.

Difference between hard and soft labels:

# hard labels
real = 1
fake = 0

# soft labels
real = np.random.uniform(0.7, 1.0)  # 1
fake = np.random.uniform(0.0, 0.3)  # 0

