How is a ResNet-50 used for deep feature extraction?
The authors use so-called embeddings, it's a form to represent the images in some meaningful vector form.
The procedure to get embedding as follows. First, keep in mind most of the popular convolutional net architectures starts with convolutional layers and then have few fully connected layers. Then do the following.

Train the full network with one-hot encoded labels as usual
Take away the last fully connected layer and use the values on the previous layer as a representation.

In the case of resnet50, you will get a 2048-value float vector. The property of a neural network is that semantically close images usually have close representation on the last layers and you could use euclidian distance to measure the similarity of images in some sense (not really, but it's another long discussion)
I don't know the paper, but I glimpsed through it. You could see in formula 4 the $x$ is embedding representation. Loss at formula 5 is cross-entropy after applying the last linear layer ($Wx + b$) and then softmax.
As a side note, you could skip the first step completely and use pretrained weights as a starting point. I.e. in pytorch you could take pretrained on ImageNet classification weights as follows
import torchvision.models as models
resnet = models.resnet50(pretrained=True)