title: BertForSequenceClassification vs. BertForMultipleChoice for sentence multi-class classification
body: <p>I'm working on a text classification problem (e.g. sentiment analysis), where I need to classify a text string into one of five classes.</p>
  <p>I just started using the <a href="https://huggingface.co/transformers/index.html" rel="noreferrer">Huggingface Transformer</a> package and BERT with PyTorch. What I need is a classifier with a softmax layer on top so that I can do 5-way classification. Confusingly, there seem to be two relevant options in the Transformer package: <a href="https://huggingface.co/transformers/model_doc/bert.html#bertforsequenceclassification" rel="noreferrer">BertForSequenceClassification</a> and <a href="https://huggingface.co/transformers/model_doc/bert.html#bertformultiplechoice" rel="noreferrer">BertForMultipleChoice</a>.</p>
  <p><strong>Which one should I use for my 5-way classification task? What are the appropriate use cases for them?</strong></p>
  <p>The documentation for <strong>BertForSequenceClassification</strong> doesn't mention softmax at all, although it does mention cross-entropy. I am not sure if this class is only for 2-class classification (i.e. logistic regression).</p>
  <blockquote>
  <p><em>Bert Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks.</em></p>
  <ul>
  <li><em><strong>labels</strong> (torch.LongTensor of shape (batch_size,), optional, defaults to None) – Labels for computing the sequence classification/regression loss. Indices should be in [0, ..., config.num_labels - 1]. If config.num_labels == 1 a regression loss is computed (Mean-Square loss), If config.num_labels &gt; 1 a classification loss is computed (Cross-Entropy).</em></li>
  </ul>
  </blockquote>
  <p>The documentation for <strong>BertForMultipleChoice</strong> mentions softmax, but the way the labels are described, it sound like this class is for multi-label classification (that is, a binary classification for multiple labels).</p>
  <blockquote>
  <p><em>Bert Model with a multiple choice classification head on top (a linear layer on top of the pooled output and a softmax) e.g. for RocStories/SWAG tasks.</em></p>
  <ul>
  <li><em><strong>labels</strong> (torch.LongTensor of shape (batch_size,), optional, defaults to None) – Labels for computing the multiple choice classification loss. Indices should be in [0, ..., num_choices] where num_choices is the size of the second dimension of the input tensors.</em></li>
  </ul>
  </blockquote>
  <p>Thank you for any help.</p>