class_docstr: Inputs:
      **input_ids**: ``torch.LongTensor`` of shape ``(batch_size, num_choices, sequence_length)``:
          Indices of input sequence tokens in the vocabulary.
          The second dimension of the input (`num_choices`) indicates the number of choices to score.
          To match pre-training, BERT input sequence should be formatted with [CLS] and [SEP] tokens as follows:
  
          (a) For sequence pairs:
  
              ``tokens:         [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP]``
              
              ``token_type_ids:   0   0  0    0    0     0       0   0   1  1  1  1   1   1``
  
          (b) For single sequences:
  
              ``tokens:         [CLS] the dog is hairy . [SEP]``
              
              ``token_type_ids:   0   0   0   0  0     0   0``
  
          Indices can be obtained using :class:`pytorch_transformers.BertTokenizer`.
          See :func:`pytorch_transformers.PreTrainedTokenizer.encode` and
          :func:`pytorch_transformers.PreTrainedTokenizer.convert_tokens_to_ids` for details.
      **token_type_ids**: (`optional`) ``torch.LongTensor`` of shape ``(batch_size, num_choices, sequence_length)``:
          Segment token indices to indicate first and second portions of the inputs.
          The second dimension of the input (`num_choices`) indicates the number of choices to score.
          Indices are selected in ``[0, 1]``: ``0`` corresponds to a `sentence A` token, ``1``
          corresponds to a `sentence B` token
          (see `BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding`_ for more details).
      **attention_mask**: (`optional`) ``torch.FloatTensor`` of shape ``(batch_size, num_choices, sequence_length)``:
          Mask to avoid performing attention on padding token indices.
          The second dimension of the input (`num_choices`) indicates the number of choices to score.
          Mask values selected in ``[0, 1]``:
          ``1`` for tokens that are NOT MASKED, ``0`` for MASKED tokens.
      **head_mask**: (`optional`) ``torch.FloatTensor`` of shape ``(num_heads,)`` or ``(num_layers, num_heads)``:
          Mask to nullify selected heads of the self-attention modules.
          Mask values selected in ``[0, 1]``:
          ``1`` indicates the head is **not masked**, ``0`` indicates the head is **masked**.
      **labels**: (`optional`) ``torch.LongTensor`` of shape ``(batch_size,)``:
          Labels for computing the multiple choice classification loss.
          Indices should be in ``[0, ..., num_choices]`` where `num_choices` is the size of the second dimension
          of the input tensors. (see `input_ids` above)
  
  Outputs: `Tuple` comprising various elements depending on the configuration (config) and inputs:
      **loss**: (`optional`, returned when ``labels`` is provided) ``torch.FloatTensor`` of shape ``(1,)``:
          Classification loss.
      **classification_scores**: ``torch.FloatTensor`` of shape ``(batch_size, num_choices)`` where `num_choices` is the size of the second dimension
          of the input tensors. (see `input_ids` above).
          Classification scores (before SoftMax).
      **hidden_states**: (`optional`, returned when ``config.output_hidden_states=True``)
          list of ``torch.FloatTensor`` (one for the output of each layer + the output of the embeddings)
          of shape ``(batch_size, sequence_length, hidden_size)``:
          Hidden-states of the model at the output of each layer plus the initial embedding outputs.
      **attentions**: (`optional`, returned when ``config.output_attentions=True``)
          list of ``torch.FloatTensor`` (one for each layer) of shape ``(batch_size, num_heads, sequence_length, sequence_length)``:
          Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads.
  
  Examples::
  
      tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
      model = BertForMultipleChoice.from_pretrained('bert-base-uncased')
      choices = ["Hello, my dog is cute", "Hello, my cat is amazing"]
      input_ids = torch.tensor([tokenizer.encode(s) for s in choices]).unsqueeze(0)  # Batch size 1, 2 choices
      labels = torch.tensor(1).unsqueeze(0)  # Batch size 1
      outputs = model(input_ids, labels=labels)
      loss, classification_scores = outputs[:2]
class_docstr_tok: ["Inputs",":","*","*","input_ids","*","*",":","``","torch.LongTensor","``","of","shape","``","(","batch_size",",","num_choices",",","sequence_length",")","``",":","Indices","of","input","sequence","tokens","in","the","vocabulary",".","The","second","dimension","of","the","input","(","`","num_choices","`",")","indicates","the","number","of","choices","to","score",".","To","match","pre-training",",","BERT","input","sequence","should","be","formatted","with","[","CLS","]","and","[","SEP","]","tokens","as","follows",":","(","a",")","For","sequence","pairs",":","``","tokens",":","[","CLS","]","is","this","jack","#","#","son","#","#","ville","?","[","SEP","]","no","it","is","not",".","[","SEP","]","``","``","token_type_ids",":","0","0","0","0","0","0","0","0","1","1","1","1","1","1","``","(","b",")","For","single","sequences",":","``","tokens",":","[","CLS","]","the","dog","is","hairy",".","[","SEP","]","``","``","token_type_ids",":","0","0","0","0","0","0","0","``","Indices","can","be","obtained","using",":","class",":","`","pytorch_transformers.BertTokenizer","`",".","See",":","func",":","`","pytorch_transformers.PreTrainedTokenizer.encode","`","and",":","func",":","`","pytorch_transformers.PreTrainedTokenizer.convert_tokens_to_ids","`","for","details",".","*","*","token_type_ids","*","*",":","(","`","optional","`",")","``","torch.LongTensor","``","of","shape","``","(","batch_size",",","num_choices",",","sequence_length",")","``",":","Segment","token","indices","to","indicate","first","and","second","portions","of","the","inputs",".","The","second","dimension","of","the","input","(","`","num_choices","`",")","indicates","the","number","of","choices","to","score",".","Indices","are","selected","in","``","[","0",",","1","]","``",":","``","0","``","corresponds","to","a","`","sentence","A","`","token",",","``","1","``","corresponds","to","a","`","sentence","B","`","token","(","see","`","BERT",":","Pre-training","of","Deep","Bidirectional","Transformers","for","Language","Understanding","`","_","for","more","details",")",".","*","*","attention_mask","*","*",":","(","`","optional","`",")","``","torch.FloatTensor","``","of","shape","``","(","batch_size",",","num_choices",",","sequence_length",")","``",":","Mask","to","avoid","performing","attention","on","padding","token","indices",".","The","second","dimension","of","the","input","(","`","num_choices","`",")","indicates","the","number","of","choices","to","score",".","Mask","values","selected","in","``","[","0",",","1","]","``",":","``","1","``","for","tokens","that","are","NOT","MASKED",",","``","0","``","for","MASKED","tokens",".","*","*","head_mask","*","*",":","(","`","optional","`",")","``","torch.FloatTensor","``","of","shape","``","(","num_heads",",",")","``","or","``","(","num_layers",",","num_heads",")","``",":","Mask","to","nullify","selected","heads","of","the","self-attention","modules",".","Mask","values","selected","in","``","[","0",",","1","]","``",":","``","1","``","indicates","the","head","is","*","*","not","masked","*","*",",","``","0","``","indicates","the","head","is","*","*","masked","*","*",".","*","*","labels","*","*",":","(","`","optional","`",")","``","torch.LongTensor","``","of","shape","``","(","batch_size",",",")","``",":","Labels","for","computing","the","multiple","choice","classification","loss",".","Indices","should","be","in","``","[","0",",","...",",","num_choices","]","``","where","`","num_choices","`","is","the","size","of","the","second","dimension","of","the","input","tensors",".","(","see","`","input_ids","`","above",")","Outputs",":","`","Tuple","`","comprising","various","elements","depending","on","the","configuration","(","config",")","and","inputs",":","*","*","loss","*","*",":","(","`","optional","`",",","returned","when","``","labels","``","is","provided",")","``","torch.FloatTensor","``","of","shape","``","(","1",",",")","``",":","Classification","loss",".","*","*","classification_scores","*","*",":","``","torch.FloatTensor","``","of","shape","``","(","batch_size",",","num_choices",")","``","where","`","num_choices","`","is","the","size","of","the","second","dimension","of","the","input","tensors",".","(","see","`","input_ids","`","above",")",".","Classification","scores","(","before","SoftMax",")",".","*","*","hidden_states","*","*",":","(","`","optional","`",",","returned","when","``","config.output_hidden_states=True","``",")","list","of","``","torch.FloatTensor","``","(","one","for","the","output","of","each","layer","+","the","output","of","the","embeddings",")","of","shape","``","(","batch_size",",","sequence_length",",","hidden_size",")","``",":","Hidden-states","of","the","model","at","the","output","of","each","layer","plus","the","initial","embedding","outputs",".","*","*","attentions","*","*",":","(","`","optional","`",",","returned","when","``","config.output_attentions=True","``",")","list","of","``","torch.FloatTensor","``","(","one","for","each","layer",")","of","shape","``","(","batch_size",",","num_heads",",","sequence_length",",","sequence_length",")","``",":","Attentions","weights","after","the","attention","softmax",",","used","to","compute","the","weighted","average","in","the","self-attention","heads",".","Examples",":",":","tokenizer","=","BertTokenizer.from_pretrained","(","'bert-base-uncased","'",")","model","=","BertForMultipleChoice.from_pretrained","(","'bert-base-uncased","'",")","choices","=","[","``","Hello",",","my","dog","is","cute","''",",","``","Hello",",","my","cat","is","amazing","''","]","input_ids","=","torch.tensor","(","[","tokenizer.encode","(","s",")","for","s","in","choices","]",")",".unsqueeze","(","0",")","#","Batch","size","1",",","2","choices","labels","=","torch.tensor","(","1",")",".unsqueeze","(","0",")","#","Batch","size","1","outputs","=","model","(","input_ids",",","labels=labels",")","loss",",","classification_scores","=","outputs","[",":2","]"]
code: def __init__(self, config):
      super(BertForReranking, self).__init__(config)
  
      self.bert = BertModel(config)
      self.dropout = nn.Dropout(config.hidden_dropout_prob)
      self.classifier = nn.Linear(config.hidden_size, 1)
      self.init_weights()
  
imports: import torch
  import os
  import numpy
  from pytorch_transformers.modeling_bert import BertPreTrainedModel, BertConfig, BertModel
  from pytorch_transformers.tokenization_bert import BertTokenizer
  from torch.utils.data import DataLoader, SequentialSampler, TensorDataset
  from torch import nn
  from torch.nn import CrossEntropyLoss, MSELoss
  from tqdm import tqdm
  from pytorch_transformers.optimization import AdamW, WarmupLinearSchedule
  from pytorch_transformers.file_utils import PYTORCH_PRETRAINED_BERT_CACHE
label_desc: Use image node_id 1 to create a new BertForReranking object from inherited base classes: BertPreTrainedModel with example: obj = BertForReranking(config)