Functionality of `detach()` in PyTorch for this specific case
I know that detach() is used for detaching a variable from the computational graph. In that context, are the following expressions x = x - torch.mean(x, dim=0).detach() and x = x - torch.mean(x, dim=0) equivalent? I just want to subtract the mean out, don't want to pass gradients through the average calculation.

If you do not detach the mean, then you have lateral dependencies between all elements in the batch (dim=0) when estimating the gradient.
