class_docstr: Give the loss a penalty with the result on the profiler.
  
  Latency losses in `TuNAS <https://arxiv.org/pdf/2008.06120.pdf>`__ and `ProxylessNAS <https://arxiv.org/pdf/1812.00332.pdf>`__
  are its special cases.
  
  The computation formula is divided into two steps,
  where we first compute a ``normalized_penalty``, whose zero point is when the penalty meets the baseline,
  and then we aggregate it with the original loss.
  
  .. math::
  
      \begin{aligned}
          \text{normalized_penalty} ={} & \text{nonlinear}(\frac{\text{penalty}}{\text{baseline}} - 1) \\
          \text{loss} ={} & \text{aggregate}(\text{original_loss}, \text{normalized_penalty})
      \end{aligned}
  
  where ``penalty`` here is the result returned by the profiler.
  
  For example, when ``nonlinear`` is ``positive`` and ``aggregate`` is ``add``, the computation formula is:
  
  .. math::
  
      \text{loss} = \text{original_loss} + \text{scale} * (max(\frac{\text{penalty}}{\text{baseline}}, 1) - 1, 0)
  
  Parameters
  ----------
  profiler
      The profiler which is used to profile the sample.
  scale
      The scale of the penalty.
  baseline
      The baseline of the penalty.
  nonlinear
      The nonlinear function to apply to :math:`\frac{\text{penalty}}{\text{baseline}}`.
      The result is called ``normalized_penalty``.
      If ``linear``, then keep the original value.
      If ``positive``, then apply the function :math:`max(0, \cdot)`.
      If ``negative``, then apply the function :math:`min(0, \cdot)`.
      If ``absolute``, then apply the function :math:`abs(\cdot)`.
  aggregate
      The aggregate function to merge the original loss with the penalty.
      If ``add``, then the final loss is :math:`\text{original_loss} + \text{scale} * \text{normalized_penalty}`.
      If ``mul``, then the final loss is :math:`\text{original_loss} * (1 + \text{normalized_penalty})^{\text{scale}}`.
class_docstr_tok: ["Give","the","loss","a","penalty","with","the","result","on","the","profiler",".","Latency","losses","in","`","TuNAS","<","https",":","\/\/arxiv.org\/pdf\/2008.06120.pdf",">","`","__","and","`","ProxylessNAS","<","https",":","\/\/arxiv.org\/pdf\/1812.00332.pdf",">","`","__","are","its","special","cases",".","The","computation","formula","is","divided","into","two","steps",",","where","we","first","compute","a","``","normalized_penalty","``",",","whose","zero","point","is","when","the","penalty","meets","the","baseline",",","and","then","we","aggregate","it","with","the","original","loss",".","..","math",":",":","\\begin","{","aligned","}","\\text","{","normalized_penalty","}","=","{","}","&","\\text","{","nonlinear","}","(","\\frac","{","\\text","{","penalty","}","}","{","\\text","{","baseline","}","}","-","1",")","\\\\","\\text","{","loss","}","=","{","}","&","\\text","{","aggregate","}","(","\\text","{","original_loss","}",",","\\text","{","normalized_penalty","}",")","\\end","{","aligned","}","where","``","penalty","``","here","is","the","result","returned","by","the","profiler",".","For","example",",","when","``","nonlinear","``","is","``","positive","``","and","``","aggregate","``","is","``","add","``",",","the","computation","formula","is",":","..","math",":",":","\\text","{","loss","}","=","\\text","{","original_loss","}","+","\\text","{","scale","}","*","(","max","(","\\frac","{","\\text","{","penalty","}","}","{","\\text","{","baseline","}","}",",","1",")","-","1",",","0",")","Parameters","--","--","--","--","--","profiler","The","profiler","which","is","used","to","profile","the","sample",".","scale","The","scale","of","the","penalty",".","baseline","The","baseline","of","the","penalty",".","nonlinear","The","nonlinear","function","to","apply","to",":","math",":","`","\\frac","{","\\text","{","penalty","}","}","{","\\text","{","baseline","}","}","`",".","The","result","is","called","``","normalized_penalty","``",".","If","``","linear","``",",","then","keep","the","original","value",".","If","``","positive","``",",","then","apply","the","function",":","math",":","`","max","(","0",",","\\cdot",")","`",".","If","``","negative","``",",","then","apply","the","function",":","math",":","`","min","(","0",",","\\cdot",")","`",".","If","``","absolute","``",",","then","apply","the","function",":","math",":","`","abs","(","\\cdot",")","`",".","aggregate","The","aggregate","function","to","merge","the","original","loss","with","the","penalty",".","If","``","add","``",",","then","the","final","loss","is",":","math",":","`","\\text","{","original_loss","}","+","\\text","{","scale","}","*","\\text","{","normalized_penalty","}","`",".","If","``","mul","``",",","then","the","final","loss","is",":","math",":","`","\\text","{","original_loss","}","*","(","1","+","\\text","{","normalized_penalty","}",")","^","{","\\text","{","scale","}","}","`","."]
code: def __init__(
      self,
      profiler: Profiler,
      baseline: float,
      scale: float = 1.0,
      *,
      nonlinear: Literal[
          "linear", "positive", "negative", "absolute"
      ] = "linear",
      aggregate: Literal["add", "mul"] = "add",
  ):
      super().__init__()
      self.profiler = profiler
      self.scale = scale
      self.baseline = baseline
      self.nonlinear = nonlinear
      self.aggregate = aggregate
  
imports: from __future__ import annotations
  import logging
  from typing import cast
  from typing_extensions import Literal
  import numpy
  import torch
  from torch import nn
  from nni.mutable import Sample
  from nni.nas.profiler import Profiler, ExpressionProfiler
  from .supermodule._expression_utils import expression_expectation
label_desc: Use image node_id 1 to create a new ProfilerPenalty object from inherited base classes: nn with example: obj = ProfilerPenalty(profiler, baseline, scale)