class_docstr: Thresholdd binarizer.
  Computes a binary mask M from a real value matrix S such that `M_{i,j} = 1` if and only if `S_{i,j} >       au`
  where `     au` is a real value threshold.
  
  Implementation is inspired from:
      https://github.com/arunmallya/piggyback
      Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
      Arun Mallya, Dillon Davis, Svetlana Lazebnik
class_docstr_tok: ["Thresholdd","binarizer",".","Computes","a","binary","mask","M","from","a","real","value","matrix","S","such","that","`","M_","{","i",",","j","}","=","1","`","if","and","only","if","`","S_","{","i",",","j","}",">","au","`","where","`","au","`","is","a","real","value","threshold",".","Implementation","is","inspired","from",":","https",":","\/\/github.com\/arunmallya\/piggyback","Piggyback",":","Adapting","a","Single","Network","to","Multiple","Tasks","by","Learning","to","Mask","Weights","Arun","Mallya",",","Dillon","Davis",",","Svetlana","Lazebnik"]
docstr: Args:
      inputs (`torch.FloatTensor`)
          The input matrix from which the binarizer computes the binary mask.
      threshold (`float`)
          The threshold value (in R).
      sigmoid (`bool`)
          If set to ``True``, we apply the sigmoid function to the `inputs` matrix before comparing to `threshold`.
          In this case, `threshold` should be a value between 0 and 1.
  Returns:
      mask (`torch.FloatTensor`)
          Binary matrix of the same size as `inputs` acting as a mask (1 - the associated weight is
          retained, 0 - the associated weight is pruned).
docstr_tok: ["Args",":","inputs","(","`","torch.FloatTensor","`",")","The","input","matrix","from","which","the","binarizer","computes","the","binary","mask",".","threshold","(","`","float","`",")","The","threshold","value","(","in","R",")",".","sigmoid","(","`","bool","`",")","If","set","to","``","True","``",",","we","apply","the","sigmoid","function","to","the","`","inputs","`","matrix","before","comparing","to","`","threshold","`",".","In","this","case",",","`","threshold","`","should","be","a","value","between","0","and","1",".","Returns",":","mask","(","`","torch.FloatTensor","`",")","Binary","matrix","of","the","same","size","as","`","inputs","`","acting","as","a","mask","(","1","-","the","associated","weight","is","retained",",","0","-","the","associated","weight","is","pruned",")","."]
code: def forward(
      ctx, inputs: torch.tensor, threshold: float, sigmoid: bool
  ):
      """
      Args:
          inputs (`torch.FloatTensor`)
              The input matrix from which the binarizer computes the binary mask.
          threshold (`float`)
              The threshold value (in R).
          sigmoid (`bool`)
              If set to ``True``, we apply the sigmoid function to the `inputs` matrix before comparing to `threshold`.
              In this case, `threshold` should be a value between 0 and 1.
      Returns:
          mask (`torch.FloatTensor`)
              Binary matrix of the same size as `inputs` acting as a mask (1 - the associated weight is
              retained, 0 - the associated weight is pruned).
      """
      nb_elems = inputs.numel()
      nb_min = int(0.005 * nb_elems) + 1
      if sigmoid:
          mask = (torch.sigmoid(inputs) > threshold).type(inputs.type())
      else:
          mask = (inputs > threshold).type(inputs.type())
      if mask.sum() < nb_min:
          # We limit the pruning so that at least 0.5% (half a percent) of the weights are remaining
          k_threshold = (
              inputs.flatten()
              .kthvalue(max(nb_elems - nb_min, 1))
              .values
          )
          mask = (inputs > k_threshold).type(inputs.type())
      return mask
  
file_path: H2O/h2o_flexgen/benchmark/third_party/transformers/examples/research_projects/movement-pruning/emmental/modules/binarizer.py
imports: import torch
  from torch import autograd
label_desc: Use image node_id 1 for calling the ThresholdBinarizer obj's underlying member method code with example usage: obj.forward(ctx, inputs, threshold, sigmoid) and returns: mask