why do we need to call `detach_` in `zero_grad`?
I am reading the source code of optimizer.zero_grad()

http://pytorch.org/docs/master/_modules/torch/optim/optimizer.html#Optimizer.zero_grad



def zero_grad(self):
    """Clears the gradients of all optimized :class:`Variable` s."""
    for group in self.param_groups:
        for p in group['params']:
            if p.grad is not None:
                p.grad.detach_()
                p.grad.zero_()


I wonder why detach_() is necessary? What does it even mean to detach a gradient variable instead of a normal variable?

Why isn't zero_() enough?

You do not want previous gradient updates to affect future gradient updates so you are detaching the previous gradients from the graph.
