math operator difference *= or +=
I found a weird thing when I used the operator e.g. *= or +=

The code:

aa = Variable(torch.FloatTensor([[1,2],[3,4]]))
bb = aa
bb = bb*2
print(bb)
print(aa)

cc = Variable(torch.FloatTensor([[1,2],[3,4]]))
dd = cc
dd *= 2
print(cc)
print(dd)


The results showed as below:

Variable containing:
 2  4
 6  8
[torch.FloatTensor of size 2x2]

Variable containing:
 1  2
 3  4
[torch.FloatTensor of size 2x2]

Variable containing:
 2  4
 6  8
[torch.FloatTensor of size 2x2]

Variable containing:
 2  4
 6  8
[torch.FloatTensor of size 2x2]


As you see, when I used bb=bb*2, aa was not be affected. However, if using dd *= 2, cc seems to point towards (share) the same address as cc, it is changed.

Their respective previous line is the same, e.g. bb = aa and dd = cc. It seems that the *= operator changed the original deep copy to shallow copy and the change was made after the copy line itself.

I am wondering if this is a bug. If it is, it is important since it affects the basic mathematics operations. Generally, I thought that just use the built-in operation functions e.g. torch.add() is a good solution.

OS: Mac OS X
PyTorch version: 3.0
How you installed PyTorch (conda, pip, source): conda
Python version: 3.6
CUDA/cuDNN version: None
GPU models and configuration: None


*---------------------------------------

I understand dd *= 2 is multiplication in-place, but how the value of dd transfer into cc? but if I used dd = dd * 2, the new values did not transfer to cc? There is no difference in the previous line: dd = cc and bb =aa.
BTW, in python (not pytorch Variable or Tensor), dd *= 2 and dd = dd * 2 both will not affect cc value.

When you do dd = cc, both dd and cc are now references to the same object (same for bb = aa). Nothing is being copied!

When you do bb = bb * 2, the * operator creates a new object and bb now refers to that object. No existing object is changed.

When you do dd *= 2, the object that dd refers to (and which cc also refers to) is changed.

So the difference is that * creates a new object and = makes a variable refer to a new object (rather than changing the object in any way), whereas *= changes the object.

It may be counter-intuitive that x *= y behaves differently than x = x * y, but those are the semantics of the language, not a bug.
