Huggingface GPT2 and T5 model APIs for sentence classification?
I've successfully used the Huggingface Transformers BERT model to do sentence classification  using the BERTForSequenceClassification class and API. I've used it for both 1-sentence sentiment analysis and 2-sentence NLI.
I can see that other models have analogous classes, e.g. XLNetForSequenceClassification and RobertaForSequenceClassification. This type of sentence classification usually involves placing a classifier layer on top of a dense vector representing the entirety of the sentence.
Now I'm trying to use the GPT2 and T5 models. However, when I look at the available classes and API for each one, there is no equivalent &quot;ForSequenceClassification&quot; class. For example, for GPT2 there are GPT2Model, GPT2LMHeadModel, and GPT2DoubleHeadsModel classes. Perhaps I'm not familiar enough with the research for GPT2 and T5, but I'm certain that both models are capable of sentence classification.
So my questions are:

What Huggingface classes for GPT2 and T5 should I use for 1-sentence classification?

What classes should I use for 2-sentence (sentence pair) classification (like natural language inference)?


Thank you for any help.

Well, why not use the code for GPT2LMHeadModel itself as an inspiration :
class MyGPT2LMHeadModel(GPT2PreTrainedModel):
    def __init__(self, config, num_classes):
        super().__init__(config)
        self.transformer = GPT2Model.from_pretrained('gpt2')
        #self.lm_head = nn.Linear(config.n_embd, config.vocab_size, bias=False)
        self.lm_head = nn.Linear(config.n_embd, num_classes, bias=False)

...

    def forward(...):
        hidden_states = self.transformer(...)[0]
        lm_logits = self.lm_head(hidden_states)
...


