The correct way to build a binary classifier for CNN
I created a neural network on pytorch using the pretraining model VGG16 and added my own extra layer to define belonging to one of two classes. For example bee or ant.

model = models.vgg16(pretrained=True)
# Freeze early layers
for param in model.parameters():
    param.requires_grad = False
n_inputs = model.classifier[6].in_features
# Add on classifier
model.classifier[6] = nn.Sequential(
    nn.Linear(n_inputs, 256), nn.ReLU(), nn.Dropout(0.2),
    nn.Linear(256, 2), nn.LogSoftmax(dim=1))


The model works well with two classes, but if to upload a crocodile image into it, it is likely to take it for a bee) 
Now I want to make a binary classifier based on this model, which defines for example a bee or not a bee (absolutely any image without a bee)
I am just starting to understand neural networks and I need advice on whether the correct way would be training in two groups of images in one of which will only bees and in the other several thousand random images. Or should it be done in another way?

It is indeed not surprising that a 2-class classifier fails with an image not belonging to any class.

To train your new one-class classifier, yes, use in our test set bees images and a set of non bee images. You need to accommodate for the imbalance between the classes as well to avoid overfitting just the bees images you have. The test accuracy would show such a bias.
