Resizing PyTorch tensor with grad to smaller size
I am trying to downsize a tensor from let's say (3,3) to (1, 1), but I want to keep the original tensor:

import torch

a = torch.rand(3, 3)
a_copy = a.clone()
a_copy.resize_(1, 1)


I need requires_grad=True in my initial tensor but PyTorch forbids this me from trying to resize the copy:

a = torch.rand(3, 3, requires_grad=True)
a_copy = a.clone()
a_copy.resize_(1, 1)


Throws an error:

Traceback (most recent call last):
  File "pytorch_test.py", line 7, in &lt;module&gt;
    a_copy.resize_(1, 1)
RuntimeError: cannot resize variables that require grad


Clone and Detach

I tried to .clone() and .detach() as well:

a = torch.rand(3, 3, requires_grad=True)
a_copy = a.clone().detach()

with torch.no_grad():
    a_copy.resize_(1, 1)


which gives this error instead:

Traceback (most recent call last):
  File "pytorch_test.py", line 14, in &lt;module&gt;
    a_copy.resize_(1, 1)
RuntimeError: set_sizes_contiguous is not allowed on a Tensor created from .data or .detach().
If your intent is to change the metadata of a Tensor (such as sizes / strides / storage / storage_offset)
without autograd tracking the change, remove the .data / .detach() call and wrap the change in a `with torch.no_grad():` block.
For example, change:
    x.data.set_(y)
to:
    with torch.no_grad():
        x.set_(y)


This behaviour had been stated in the docs and #15070.

With no_grad()

So, following what they said in the error message, I removed .detach() and used no_grad() instead:

a = torch.rand(3, 3, requires_grad=True)
a_copy = a.clone()

with torch.no_grad():
    a_copy.resize_(1, 1)


But it still gives me an error about grad:

Traceback (most recent call last):
  File "pytorch_test.py", line 21, in &lt;module&gt;
    a_copy.resize_(1, 1)
RuntimeError: cannot resize variables that require grad


Similar questions

I have looked at Resize PyTorch Tensor but it the tensor in that example retains all original values.
I have also looked at Pytorch preferred way to copy a tensor which is the method I am using to copy the tensor.

I am using PyTorch version 1.4.0

I think you should first detach and then clone:

a = torch.rand(3, 3, requires_grad=True)
a_copy = a.detach().clone()
a_copy.resize_(1, 1)


Note: a.detach() returns a new tensor detached from the current graph (it doesn't detach a itself from the graph as a.detach_() does). But because it shares storage with a, you also should clone it. In this way, whatever you do with the a_copy won't affect a. However I am not sure why a.detach().clone() works but a.clone().detach() gives error.

Edit:

The following code also works (which is probably a better solution):

a = torch.rand(3, 3, requires_grad=True)

with torch.no_grad():
    a_copy = a.clone()
    a_copy.resize_(1, 1)

