Does pytorch broadcast consume less memory than expand?
Does pytorch operations using broadcast consume less memory than expand? For example, are the following two programs different in memory usage?
import torch
x = torch.randn(20,1)
y = torch.randn(1,20)
z = x*y

import torch
x = torch.randn(20,1).expand(-1,20)
y = torch.randn(1,20).expand(20,-1)
z = x*y


According to the documentation page of torch.expand:

Expanding a tensor does not allocate new memory, but only creates a new view on the existing tensor

You can experiment it yourself by profiling the calls (here in Colab):
&gt;&gt;&gt; x = torch.randn(200,1)
&gt;&gt;&gt; y = torch.randn(1,200)
&gt;&gt;&gt; %memit z = x*y
peak memory: 286.85 MiB, increment: 0.31 MiB

&gt;&gt;&gt; x = torch.randn(200,1).expand(-1,200)
&gt;&gt;&gt; y = torch.randn(1,200).expand(200,-1)
&gt;&gt;&gt; %memit z = x*y
peak memory: 286.86 MiB, increment: 0.00 MiB


%memit is a magic function provided by memory_profiler:
pip install memory_profiler

%load_ext memory_profiler

