Why am I able to change the value of a tensor without the computation graph knowing about it in Pytorch with detach?
I can change the value of a tensor that requires grad without autograd knowing about it:

def error_unexpected_way_to_by_pass_safety():
    import torch 
    a = torch.tensor([1,2,3.], requires_grad=True)
    # are detached tensor's leafs? yes they are
    a_detached = a.detach()
    #a.fill_(2) # illegal, warns you that a tensor which requires grads is used in an inplace op (so it won't be recorded in computation graph so it wont take the right derivative of the forward path as this op won't be in it)
    a_detached.fill_(2) # weird that this one is allowed, seems to allow me to bypass the error check from the previous comment...?!
    print(f'a = {a}')
    print(f'a_detached = {a_detached}')
    a.sum().backward()


this throws no errors. Though, I am able to change the contents of a which is a tensor that requires grad without autograd knowing about. Which means the computation graph does not know about this op (filling with 2). This seems wrong. Can anyone shed light what is going on?

.detach gives you a view on the same data, so modifying the data of the detached tensor modifies the data of the original. You can check this like so:

a.data_ptr() == a_detached.data_ptr() # True


As for why this is how .detach is implemented (as opposed to doing a defensive copy), that's a design question that only the PyTorch authors know the answer to. I assume that it's to save unnecessary copies, but users then need to be aware that they have to copy the tensors themselves if they want to modify the detached ones in-place.

Note that you can also alter the non-detached tensor if you really want to:

a.data.fill_(2)


PyTorch isn't trying to stop you from "hacking" autograd; users still have to be aware of how to use tensors properly so that gradients will be tracked correctly.
