class_docstr: Rescores source and target tokens based on a model
docstr: logprobs have the log probabilities for each possible token
  for each hypothesis. here, we extract the logprobs matching the
  target tokens.
docstr_tok: ["logprobs","have","the","log","probabilities","for","each","possible","token","for","each","hypothesis",".","here",",","we","extract","the","logprobs","matching","the","target","tokens","."]
code: def compute_scores(
      self, tgt_tokens, logprobs, possible_translation_tokens=None
  ):
      """logprobs have the log probabilities for each possible token
      for each hypothesis. here, we extract the logprobs matching the
      target tokens.
      """
  
      def clean_tgt_tokens(tgt_tokens, possible_translation_tokens):
          for i, hypo_tokens in enumerate(tgt_tokens):
              for j, hypo_token in enumerate(hypo_tokens):
                  tgt_tokens[i][j] = (
                      possible_translation_tokens == hypo_token
                  ).nonzero()
          return tgt_tokens
  
      if possible_translation_tokens is not None:
          tgt_tokens = clean_tgt_tokens(
              tgt_tokens, possible_translation_tokens
          )
  
      tgt_tokens = tgt_tokens[:, 1:]  # get rid of initial eos token
  
      i, j = torch.meshgrid(
          torch.arange(0, tgt_tokens.size(0)).long(),
          torch.arange(0, tgt_tokens.size(1)).long(),
      )
      hypos_tokens_probs = torch.zeros(tgt_tokens.shape)
      hypos_tokens_probs = logprobs[:, :-1, :][i, j, tgt_tokens]
      pad = self.task.target_dictionary.pad()
      mask = 1.0 - tgt_tokens.eq(pad).float()
      hypos_scores = (hypos_tokens_probs * mask).sum(dim=1)
      return hypos_scores
  
imports: import torch
  from pytorch_translate import utils
label_desc: Use image node_id 5 for calling the SimpleModelScorer obj's underlying member method code with example usage: obj.compute_scores(tgt_tokens, logprobs, possible_translation_tokens) and returns: hypos_scores, tgt_tokens