how effective is transfer learning? keeping only two specific output features without resetting features
I want to keep only two specific output features without resetting features.
Resetting features would lose the pre-trained weights.
For example, I don't want to do...
# https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html?highlight=transfer%20learning%20ant%20bees

model_ft = models.resnet18(pretrained=True)
num_ftrs = model_ft.fc.in_features
model_ft.fc = nn.Linear(num_ftrs, 2)

Here is code (following the transfer learning tutorial on Pytorch)
I want to do this to see how effective transfer learning is.
Even without transfer learning, a model might be effective. Removing 998 out of 1000 categories and leaving only two categories, ant and bee, could be a great categorical model since you are left with only two choices.
I do not want to re-train the model, I want to use the weights as it is, otherwise, it will be the same as transfer learning.

You can certainly try this. You can reduce the model output to just the two logits you want to compare with:
                chosen_cats = torch.Tensor([ant_index, bee_index]).long()
                with torch.set_grad_enabled(phase == 'train'):
                outputs = model(inputs)
                outputs = torch.index_select(output, 1, chosen_cats)
                _, preds = torch.max(outputs, 1)
                loss = criterion(outputs, labels)

In this scenario, the preds will be 0 or 1, with 0 predicting ant and 1 predicting bee, so you will need to also modify your labels to reflect this.
