what is the difference between unsqueez_ in pytorch and epxand_dim in keras and what will be the shape of output after using it?
I am a beginner in keras and I have a pytorch code that I need to change it to keras, but I could not understand some part of it. specially I have problems in the size of the output shape.  the shape of image is (:, 3,32,32) and the first dimension of image is the size of the batch. now, my question is: what this line do and what is the output shape:

    image_yuv_ch = image[:, channel, :, :].unsqueeze_(1)


it adds a dimension in position 1? what is the output shape?:(
the size of filters was (64,8,8) and then we have filters.unsqueez_(1), is this means the new shape of filters is (64,1,8,8)?
what does this line do?  image_conv = F.conv2d(image_yuv_ch, filters, stride=8) is it the same as conv2d in keras what is the shape of output tensor from it? I also could not understand what view do? I know it tries to show tensor in new shape but in the below code I could not understand the output shape after each unsqueez_, permute or view. could you please tell me what is the output shape of each line? Thank you in advance.

import torch.nn.functional as F
def apply_conv(self, image, filter_type: str):



        if filter_type == 'dct':
            filters = self.dct_conv_weights
        elif filter_type == 'idct':
            filters = self.idct_conv_weights
        else:
            raise('Unknown filter_type value.')

        image_conv_channels = []
        for channel in range(image.shape[1]):
            image_yuv_ch = image[:, channel, :, :].unsqueeze_(1)
            image_conv = F.conv2d(image_yuv_ch, filters, stride=8)
            image_conv = image_conv.permute(0, 2, 3, 1)
            image_conv = image_conv.view(image_conv.shape[0], image_conv.shape[1], image_conv.shape[2], 8, 8)
            image_conv = image_conv.permute(0, 1, 3, 2, 4)
            image_conv = image_conv.contiguous().view(image_conv.shape[0],
                                                  image_conv.shape[1]*image_conv.shape[2],
                                                  image_conv.shape[3]*image_conv.shape[4])

            image_conv.unsqueeze_(1)

            # image_conv = F.conv2d()
            image_conv_channels.append(image_conv)

        image_conv_stacked = torch.cat(image_conv_channels, dim=1)

        return image_conv_stacked


It seems like you are Keras-user or Tensorflow-user and trying to learn Pytorch. 
You should go to the website of Pytorch document to understand more about each operation.


unsqueeze is to expand the dim by 1 of the tensor. The underscore in unsqueeze_() means this is in-place function.  
view() can be understood as .reshape() in keras.
permute() is to switch multiple dimensions of tensor. For example:


x = torch.randn(1,2,3) # shape [1,2,3]
x = torch.permute(2,0,1) # shape [3,1,2]


In order to know the shape of the tensor after each operation, just simply add print(x.size()). For example:

image_conv = image_conv.permute(0, 2, 3, 1)
print(image_conv.size())

image_conv = image_conv.view(image_conv.shape[0], image_conv.shape[1], 
print(image_conv.size())

image_conv.shape[2], 8, 8)
print(image_conv.size())

image_conv = image_conv.permute(0, 1, 3, 2, 4)
print(image_conv.size())



The big difference between Pytorch and Tensorflow (back-end of Keras) is that Pytorch will generate a dynamic graph, rather than a static graph as Tensorflow. Your way of defining a model would not work properly in Pytorch since the weights of conv will not be save in model.parameters() which can't be optimized during the backpropagation.

One more comment, please check this link to learn how to define a proper model using Pytorch:

import torch.nn as nn
import torch.nn.functional as F
class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
       x = F.relu(self.conv1(x))
       return F.relu(self.conv2(x))



The code for the comment:


import torch

x = torch.randn(8, 3, 32, 32)
print(x.shape)
torch.Size([8, 3, 32, 32])
channel = 1
y = x[:, channel, :, :]
print(y.shape)
torch.Size([8, 32, 32])

y = y.unsqueeze_(1)
print(y.shape)
torch.Size([8, 1, 32, 32])


Hope this helps and enjoy your learning!
