PyTorch conversion between tensor and numpy array: the addition operation
I am following the 60-minute blitz on PyTorch but have a question about conversion of a numpy array to a tensor. Tutorial example here.
This piece of code:
import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
np.add(a, 1, out=a)
print(a)
print(b)

yields

[2. 2. 2. 2. 2.]
tensor([2., 2., 2., 2., 2.], dtype=torch.float64)

However
import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
a = a + 1 #the diff is here 
print(a)
print(b)

yields

[2. 2. 2. 2. 2.]
tensor([1., 1., 1., 1., 1.], dtype=torch.float64)

Why are the outputs different?

This actually has little to do with PyTorch. Compare

import numpy as np
a = np.ones(5)
b = a


followed by either

np.add(a, 1, out=a)
print(b)


or

a = a + 1
print(b)


There is a difference between np.add(a, 1, out=a) and a = a + 1. In the former you retain the same object (array) a with different values (2 instead of 1); in the latter you get a new array, which is bound to the same variable name a and has values of 2. However, the "original" a is discarded and unless something else (b) points to it, would be deallocated. In other words, the first operation is in-place and the latter out-of-place. Since b holds on to the array originally found at a, reassigning a + 1 to a does not affect the value of b. An alternative in-place mutation syntax would be

a[:] = a + 1
print(b)


Regarding PyTorch, it's very simple. from_numpy creates a tensor which aliases an actual object (array), so it is equivalent to the b = a line in my first snippet. The tensor will track the changes in the array named a at the point of calling, rather than the changes of what the name a points to.
