Fine-tuning Resnet using pre-convoluted features
As we know that convolution layers are expensive to calculate , I would like to compute the output of the convolution layers once and use them to train the fully connected layer of my Resnet, in order to speed up the process. 

In the case of a VGG model, we can compute the output of the first convolutional part as follows 

x = model_vgg.features(inputs)


But how can I do to extract features from a Resnet?

Thanks in advance

I guess you can try hacking through the net. I'll use resnet18 as an example:

import torch
from torch import nn
from torchvision.models import resnet18

net = resnet18(pretrained=False)
print(net)


You'll see something like:

....
      (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
      (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
  )
  (avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
  (fc): Linear(in_features=512, out_features=1000, bias=True)


Let's store the Linear layer somewhere, and in its place, put a dummy layer. Then the output of the whole net is actually the output of the conv layers.

x = torch.randn(4,3,32,32) # dummy input of batch size 4 and 32x32 rgb images
out = net(x)
print(out.shape)
&gt;&gt;&gt; 4, 1000 # batch size 4, 1000 default class predictions

store_fc = net.fc      # Save the actual Linear layer for later
net.fc = nn.Identity() # Add a layer which actually does nothing
out = net(x)
print(out.shape)
&gt;&gt;&gt; 4, 512 # batch size 4, 512 features that are the input to the actual fc layer.

