When using torch.backward() for a GANs generator, why doesn't discriminator losses change in Pytorch?
My understanding of GANs is:

When training your generator, you need to back-propagate through the discriminator first so you can follow the chain rule. As a result, we can't use a .detach()  when working on our generators loss calculation.

When updating discriminator, since your generator weight update doesn't affect discriminator weight updates, we can .detach() the generator output from your computation, my understanding tells me that since this generator is no longer a part of computation graph, we don't update it during back prop any longer.


So when we update your discriminator losses with:
disc_loss.backward(retain_graph=True) at each mini-batch, we don't have to worry about your generator being part of the pipeline thanks to the .detach() function call.
But what about when we work on our generator? What is keeping our model from constantly changing our discriminator weights in accordance to generator? That isn't what we want after all right? The discriminator shouldn't learn when it is told that fake samples are real.
Why does such a model work in the first place

backward doesn't update the weights, it updates the gradients of the weights. Updating weights is the responsibility of the optimizer(s). There are different ways to implement GANs, but often you would have two optimizers, one that is responsible for updating the weights (and resetting the gradients) of the generator and one that is responsible for updating the weights (and resetting the gradients) of the discriminator. Upon initialization, each optimizer is provided only the weights of the model it will update. Therefore, when you call an optimizer's step method it only updates those weights. Using separate optimizers is what prevents the discriminator weights from being updated while minimizing the generator's loss function.
