RuntimeError with copy_() and expand() - Unsupported operation: more than one element of the written-to tensor refers to a single memory location
I'm migrating a repository from Pytorch Nightly 1.0.0 to 1.3.1. Stripping the unnecessary details, it is basically performing the following sequence of operations:

mu = torch.tensor(0.005)
bar = torch.eye(5, 5)
foo = torch.eye(5).expand(5, 5, 5)

# update
bar.copy_(mu * bar)  # ok!
foo.copy_(mu * foo)  # error


bar.copy_(mu * bar) works, while when I try to foo.copy_() the result, it gives the following error:


  RuntimeError: unsupported operation: more than one element of the written-to tensor refers to a single memory location. Please clone() the tensor before performing the operation.


This is because expand() only creates a new view on the existing tensor, thus it doesn't allocate the full memory necessary to receive all the elements from the operation mu * foo, which has more elements than the original tensor foo. You can fix it by either using expand().clone() or repeat(), which will give you the full tensor.

foo = torch.eye(5).expand(5, 5, 5).clone()  # clone gives the full tensor
foo.copy_(mu * foo)  # ok!


albanD suggests that doing expand().clone() might still be faster than repeat().

See here and here for more details about expand() and repeat().
