class_docstr: Implements a combined loss from translation and target words prediction.
  
  References:
    [1] https://research.google.com/pubs/archive/37180.pdf
    [2] https://arxiv.org/abs/1708.01771
    [3] https://discuss.pytorch.org/t/writing-warp-loss-layer/3715
  
  TODO(T38581791):
    Although this implementation is faster than [3] (3s/it vs 70s/it), it is
    still much slower than `WordPredictionCriterion`.
    Need to explore ways to speed it up -- it looks like the training time
    increases after each iteration. This happens because it becomes harder
    to search after each update.
class_docstr_tok: ["Implements","a","combined","loss","from","translation","and","target","words","prediction",".","References",":","[","1","]","https",":","\/\/research.google.com\/pubs\/archive\/37180.pdf","[","2","]","https",":","\/\/arxiv.org\/abs\/1708.01771","[","3","]","https",":","\/\/discuss.pytorch.org\/t\/writing-warp-loss-layer\/3715","TODO","(","T38581791",")",":","Although","this","implementation","is","faster","than","[","3","]","(","3s\/it","vs","70s\/it",")",",","it","is","still","much","slower","than","`","WordPredictionCriterion","`",".","Need","to","explore","ways","to","speed","it","up","--","it","looks","like","the","training","time","increases","after","each","iteration",".","This","happens","because","it","becomes","harder","to","search","after","each","update","."]
docstr: Implements the WARP loss given in [1].
  
  In its core the function computes the following:
      loss = (X-1)/N*(xn_i - xp),
  where `xn_i` is confidence of the ith false positive, and `xp` is the
  true positive confidence. `X` is the total number of labels and `N` is
  the number of steps that it takes to find a false positive.
  Note: We might want to use ln((X-1)/N), in case N << X, which would
        expolode the loss.
  
  Args:
      prediction: Prediction that was made by the model of shape
                  [BATCH_SIZE, N_LABELS]
      target: Expected result of shape [BATCH_SIZE, N_OUTPUT_TOKENS]
      rank_weight_type: Argument to set the ranks of the weights.
                        See `set_rank_weights` for more details.
  
  Returns:
      loss: Loss as a torch.Variable
docstr_tok: ["Implements","the","WARP","loss","given","in","[","1","]",".","In","its","core","the","function","computes","the","following",":","loss","=","(","X-1",")","\/N","*","(","xn_i","-","xp",")",",","where","`","xn_i","`","is","confidence","of","the","ith","false","positive",",","and","`","xp","`","is","the","true","positive","confidence",".","`","X","`","is","the","total","number","of","labels","and","`","N","`","is","the","number","of","steps","that","it","takes","to","find","a","false","positive",".","Note",":","We","might","want","to","use","ln","(","(","X-1",")","\/N",")",",","in","case","N","<","<","X",",","which","would","expolode","the","loss",".","Args",":","prediction",":","Prediction","that","was","made","by","the","model","of","shape","[","BATCH_SIZE",",","N_LABELS","]","target",":","Expected","result","of","shape","[","BATCH_SIZE",",","N_OUTPUT_TOKENS","]","rank_weight_type",":","Argument","to","set","the","ranks","of","the","weights",".","See","`","set_rank_weights","`","for","more","details",".","Returns",":","loss",":","Loss","as","a","torch.Variable"]
code: def predictor_loss_function(
      self, prediction, target, rank_weights_type="uniform"
  ):
      """Implements the WARP loss given in [1].
  
      In its core the function computes the following:
          loss = (X-1)/N*(xn_i - xp),
      where `xn_i` is confidence of the ith false positive, and `xp` is the
      true positive confidence. `X` is the total number of labels and `N` is
      the number of steps that it takes to find a false positive.
      Note: We might want to use ln((X-1)/N), in case N << X, which would
            expolode the loss.
  
      Args:
          prediction: Prediction that was made by the model of shape
                      [BATCH_SIZE, N_LABELS]
          target: Expected result of shape [BATCH_SIZE, N_OUTPUT_TOKENS]
          rank_weight_type: Argument to set the ranks of the weights.
                            See `set_rank_weights` for more details.
  
      Returns:
          loss: Loss as a torch.Variable
      """
      batch_size = prediction.size()[0]
      n_labels = prediction.size()[1]
      n_output_tokens = target.size()[1]
      max_num_trials = n_labels - 1
  
      self.set_rank_weights(n_labels, rank_weights_type)
  
      loss = maybe_cuda(torch.zeros(batch_size, n_output_tokens))
  
      for i in range(batch_size):
          for j in range(n_output_tokens):
              target_idx = target[i, j]
              neg_labels_idx = maybe_cuda(
                  torch.tensor(
                      list(
                          set(range(n_labels))
                          - set(target[i, :].cpu().numpy())
                      )
                  )
              )
              neg_idx = torch.multinomial(neg_labels_idx.double(), 1)
              # This is the hinge loss:
              # sample_score_margin = \
              #   1 - prediction[i, target_idx] + prediction[i, neg_idx]
              # TODO:
              #   Since |- prediction[i, target_idx] + prediction[i, neg_idx]|
              #   is normally around 0.01, directly using log probability in
              #   hinge loss causes most N to be 1, thus is not a good choice.
              # Observation: translation_loss is normally ~10, similar to
              #              log_probs.
              # Alternatives: scale up score difference by 100 times to match
              #               the magnitude of 1, but we also need to consider
              #               magnitude of weights and loss;
              sample_score_margin = (
                  -prediction[i, target_idx] + prediction[i, neg_idx]
              )
              N = 1
              while sample_score_margin < 0 and N < max_num_trials:
                  neg_idx = torch.multinomial(
                      neg_labels_idx.double(), 1
                  )
                  N += 1
                  sample_score_margin = (
                      -prediction[i, target_idx]
                      + prediction[i, neg_idx]
                  )
  
              k = torch.floor(torch.tensor(max_num_trials / N)).int()
              weights = torch.sum(self.rank_weights[:k])
              score_margins = (
                  -prediction[i, target_idx] + prediction[i, neg_idx]
              )
              loss[i, j] = (
                  weights * score_margins.clamp(min=0.0)
              ).mean()
      return loss
  
imports: import abc
  import math
  import torch
  from fairseq import utils
  from fairseq.criterions import register_criterion
  from fairseq.criterions.label_smoothed_cross_entropy import LabelSmoothedCrossEntropyCriterion
  from pytorch_translate.utils import maybe_cuda
label_desc: Use image node_id 3 for calling the WARPLossCriterion obj's underlying member method code with example usage: obj.predictor_loss_function(prediction, target, rank_weights_type) and returns: loss