class_docstr: Top-k Binarizer.
  Computes a binary mask M from a real value matrix S such that `M_{i,j} = 1` if and only if `S_{i,j}`
  is among the k% highest values of S.
  
  Implementation is inspired from:
      https://github.com/allenai/hidden-networks
      What's hidden in a randomly weighted neural network?
      Vivek Ramanujan*, Mitchell Wortsman*, Aniruddha Kembhavi, Ali Farhadi, Mohammad Rastegari
class_docstr_tok: ["Top-k","Binarizer",".","Computes","a","binary","mask","M","from","a","real","value","matrix","S","such","that","`","M_","{","i",",","j","}","=","1","`","if","and","only","if","`","S_","{","i",",","j","}","`","is","among","the","k","%","highest","values","of","S.","Implementation","is","inspired","from",":","https",":","\/\/github.com\/allenai\/hidden-networks","What","'s","hidden","in","a","randomly","weighted","neural","network","?","Vivek","Ramanujan","*",",","Mitchell","Wortsman","*",",","Aniruddha","Kembhavi",",","Ali","Farhadi",",","Mohammad","Rastegari"]
docstr: Args:
      inputs (`torch.FloatTensor`)
          The input matrix from which the binarizer computes the binary mask.
      threshold (`float`)
          The percentage of weights to keep (the rest is pruned).
          `threshold` is a float between 0 and 1.
  Returns:
      mask (`torch.FloatTensor`)
          Binary matrix of the same size as `inputs` acting as a mask (1 - the associated weight is
          retained, 0 - the associated weight is pruned).
docstr_tok: ["Args",":","inputs","(","`","torch.FloatTensor","`",")","The","input","matrix","from","which","the","binarizer","computes","the","binary","mask",".","threshold","(","`","float","`",")","The","percentage","of","weights","to","keep","(","the","rest","is","pruned",")",".","`","threshold","`","is","a","float","between","0","and","1",".","Returns",":","mask","(","`","torch.FloatTensor","`",")","Binary","matrix","of","the","same","size","as","`","inputs","`","acting","as","a","mask","(","1","-","the","associated","weight","is","retained",",","0","-","the","associated","weight","is","pruned",")","."]
code: def forward(ctx, inputs: torch.tensor, threshold: float):
      """
      Args:
          inputs (`torch.FloatTensor`)
              The input matrix from which the binarizer computes the binary mask.
          threshold (`float`)
              The percentage of weights to keep (the rest is pruned).
              `threshold` is a float between 0 and 1.
      Returns:
          mask (`torch.FloatTensor`)
              Binary matrix of the same size as `inputs` acting as a mask (1 - the associated weight is
              retained, 0 - the associated weight is pruned).
      """
      # Get the subnetwork by sorting the inputs and using the top threshold %
      mask = inputs.clone()
      _, idx = inputs.flatten().sort(descending=True)
      j = int(threshold * inputs.numel())
  
      # flat_out and mask access the same memory.
      flat_out = mask.flatten()
      flat_out[idx[j:]] = 0
      flat_out[idx[:j]] = 1
      return mask
  
file_path: transformers/examples/research_projects/movement-pruning/emmental/modules/binarizer.py
imports: import torch
  from torch import autograd
label_desc: Use image node_id 1 for calling the TopKBinarizer obj's underlying member method code with example usage: obj.forward(ctx, inputs, threshold) and returns: mask