What does num_labels actually do?
When training a BERT based model one can set num_labels
AutoConfig.from_pretrained(BERT_MODEL_NAME, num_labels=num_labels)
So for example if we want to have a prediction of 3 values we may use num_labels=3.
My question is what does it do internally? Is it just connecting a nn.Linear to the last embedding layer?
Thanks

I suppose if there is a num label then the model is used for classification then simply you can go to the documentation of BERT on hugging face then search for the classification class and take a look into the code, then you will find the following:
https://github.com/huggingface/transformers/blob/bd469c40659ce76c81f69c7726759d249b4aef49/src/transformers/models/bert/modeling_bert.py#L1572
        if labels is not None:
        if self.config.problem_type is None:
            if self.num_labels == 1:
                self.config.problem_type = &quot;regression&quot;
            elif self.num_labels &gt; 1 and (labels.dtype == torch.long or labels.dtype == torch.int):
                self.config.problem_type = &quot;single_label_classification&quot;
            else:
                self.config.problem_type = &quot;multi_label_classification&quot;

        if self.config.problem_type == &quot;regression&quot;:
            loss_fct = MSELoss()
            if self.num_labels == 1:
                loss = loss_fct(logits.squeeze(), labels.squeeze())
            else:
                loss = loss_fct(logits, labels)
        elif self.config.problem_type == &quot;single_label_classification&quot;:
            loss_fct = CrossEntropyLoss()
            loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
        elif self.config.problem_type == &quot;multi_label_classification&quot;:
            loss_fct = BCEWithLogitsLoss()
            loss = loss_fct(logits, labels)

so the number of labels as we see affects using the loss function
hope this answers your question
