Upsampling Only the Last Two Dimensions of a 5D Tensor
I have a 5D tensor x (frames of a video) and I want to upsample the spatial size (the last two dimensions) of this tensor but when I use upsampling, the last three dimensions of the tensor are upsampled. For upsampling I use the following class:
class Upsample(nn.Module):
    def __init__(self, scale_factor, mode, align_corners=False):
        self.interp = interpolate
        self.scale_factor = scale_factor
        self.mode = mode
        self.align_corners=align_corners

    def forward(self, x):
        x = self.interp(x, scale_factor=self.scale_factor, mode=self.mode)
        return x

And for example, the main class that I want to upsample a 5D tensor is as follows (I condensed my code):
class Main(nn.Module):
   def __init__(self):
       super(Main, self).__init__()
       self.upsample = Upsample(scale_factor=2, mode='trilinear')

   def forward(self, x):
        x = self.upsample(x)
        return x

To be clearer, for example by applying upsampling on a tensor of x=(2,4,3,10,20), the outcome based on the aforementioned class is x=(2,4,6,20,40) but I need to have x=(2,4,3,20,40).
What is the problem and how can I solve this?

The trilinear mode of pytorch's interpolate function only supports interpolation of 5D tensor including your third dimension. If you don't mind in resizing your input tensor, you may reduce the dimension and apply bicubic mode for interpolation.
class Upsample(nn.Module):
    def __init__(self, scale_factor, mode = 'bicubic', align_corners=False):
        self.interp = interpolate
        self.scale_factor = scale_factor
        self.mode = mode
        self.align_corners=align_corners

    def forward(self, x):
        B, C, T, W, H = x.size() 
        x = x.reshape(B, C*T, W, H)
        x = self.interp(x, scale_factor=self.scale_factor, mode=self.mode)
        x = x.reshape(B, C, T, W, H)
        return x

