docstr: Calibrate a torch.nn.Module for post-training quantization.
  
  :param model:           A model to be quantized. Model type should be an instance of
                          torch.nn.Module.
  :param precision:       Global precision of quantized model,
                          supported type: 'int8', 'bf16', 'fp16', defaults to 'int8'.
  :param accelerator:     Use accelerator 'None', 'onnxruntime', 'openvino', 'jit', defaults
                          to None. None means staying in pytorch.
  :param use_ipex:        Whether we use ipex as accelerator for inference.
                          If precision != bf16, it will be ignored. Default: ``False``.
  :param calib_data:      Calibration data is required for static quantization.
                          It's also used as validation dataloader.
                          calib_data support following formats:
  
                          | 1. a torch.utils.data.dataloader.DataLoader object for training.
                          |
                          | 2. a single torch.Tensor used for training, this case is used
                          | to accept single sample input x.
                          |
                          | 3. a tuple of torch.Tensor which used for training, this case is
                          | used to accept single sample input (x, y) or (x1, x2) et al.
  :param calib_dataloader:    A torch.utils.data.dataloader.DataLoader object for calibration.
                              Required for static quantization.
                              It's also used as validation dataloader.
  
         .. warning::
            ``calib_dataloader`` will be deprecated in future release.
  
            Please use ``calib_data`` instead.
  :param eval_func:       A evaluation function which only accepts model as input and return
                          evaluation value. This parameter provides a higher degree of
                          freedom than using eval_loader and metric. Default to None meaning
                          no performance tuning, but it would be better give an evaluation
                          function to get better quantization performance.
  :param metric:              A torchmetrics.metric.Metric object for evaluation.
  :param accuracy_criterion:  Tolerable accuracy drop, defaults to None meaning no
                              accuracy control.
                              accuracy_criterion = {'absolute':0.99, 'higher_is_better':False}
                              means accuracy loss must be smaller than 0.99. For example, if
                              higher_is_better is True, then this requires original metric
                              value subtract current metric value be smaller than 0.99.
                              For inc 1.x, this value must be set to [0, 1), for inc 2.x,
                              there is no limit.
                              accuracy_criterion = {'relative':0.1, 'higher_is_better':True}
                              allows relative accuracy loss: 10%.
  :param approach:    'static' or 'dynamic'.
                      'static': post_training_static_quant,
                      'dynamic': post_training_dynamic_quant.
                      Default: 'static'. OpenVINO supports static mode only.
  :param method:          Method to do quantization. When accelerator=None, supported
      methods: 'fx', 'eager', 'ipex', defaults to 'fx'. If you don't use ipex, suggest using
      'fx' which executes automatic optimizations like fusion. For more information, please
      refer to https://pytorch.org/docs/stable/quantization.html#eager-mode-quantization.
      When accelerator='onnxruntime', supported methods: 'qlinear', 'integer', defaults
      to 'qlinear'. Suggest 'qlinear' for lower accuracy drop if using static quantization.
      More details in https://onnxruntime.ai/docs/performance/quantization.html.
      This argument doesn't take effect for OpenVINO, don't change it for OpenVINO.
  :param conf:        A path to conf yaml file for quantization.
                      Default: None, using default config.
  :param tuning_strategy:    'bayesian', 'basic', 'mse', 'sigopt'. Default: 'bayesian'.
  :param timeout:     Tuning timeout (seconds). Default: None,  which means early stop.
                      Combine with max_trials field to decide when to exit.
  :param max_trials:  Max tune times. Default: None, which means no tuning.
                      Combine with timeout field to decide when to exit.
                      "timeout=0, max_trials=1" means it will try quantization only once and
                      return satisfying best model.
  :param input_sample:      An input example to convert pytorch model into ONNX/OpenVINO/JIT.
  :param channels_last: Whether use channels last memory format, i.e. NHWC (batch size,
                        height, width, channels), as an alternative way to store tensors in
                        classic/contiguous NCHW order, only valid when precision='bf16',
                        otherwise will be ignored. This setting only works for 4-dim Tensor.
                        Default: ``False``.
  :param thread_num: (optional) An int represents how many threads(cores) is needed for
                     inference. This parameter only controls the usage of thread number in
                     later inference process of your obtained accelerated model. In other
                     words, the process of model conversion won't be restricted by this
                     parameter.
  :param device: (optional) A string represents the device of the inference. Default to 'CPU',
                  only valid when accelerator='openvino', otherwise will be ignored.
                  'CPU', 'GPU' and 'VPUX' are supported for now.
  :param onnxruntime_session_options: The session option for onnxruntime, only valid when
                                      accelerator='onnxruntime', otherwise will be ignored.
  :param openvino_config: The config to be inputted in core.compile_model. Only valid when
                          accelerator='openvino', otherwise will be ignored.
  :param simplification: whether we use onnxsim to simplify the ONNX model, only valid when
                         accelerator='onnxruntime', otherwise will be ignored. If this option
                         is set to True, new dependency 'onnxsim' need to be installed.
  :param jit_strict: Whether recording your mutable container types. This parameter will be
                     passed to ``torch.jit.trace``. if ``accelerator != 'jit'`` or
                     ``jit_method='script'``, it will be ignored. Default to True.
  :param jit_method: Whether to use ``jit.trace`` or ``jit.script`` to convert a model
                     to TorchScript. Accepted values are ``'trace'``, ``'script'``,
                     and ``None``. Default to be ``None`` meaning the try-except logic
                     to use ``jit.trace`` or ``jit.script``. If ``accelerator != 'jit'``,
                     this parameter will be ignored.
  :param dynamic_axes: dict or boolean, default to True. By default the exported onnx model
                       will have the first dim of each Tensor input as a dynamic batch_size.
                       If dynamic_axes=False, the exported model will have the shapes of all
                       input and output tensors set to exactly match those given in
                       input_sample. To specify axes of tensors as dynamic (i.e. known only
                       at run-time), set dynamic_axes to a dict with schema:
  
                       | KEY (str): an input or output name. Each name must also be provided
                       | in input_names or output_names.
                       |
                       | VALUE (dict or list): If a dict, keys are axis indices and values
                       | are axis names. If a list, each element is an axis index.
  
                       If accelerator != 'openvino'/'onnxruntime', it will be ignored.
  :param sample_size: (optional) a int represents how many samples will be used for
                      Post-training Optimization Tools (POT) from OpenVINO toolkit,
                      only valid for accelerator='openvino'. Default to 100.
                      The larger the value, the more accurate the conversion,
                      the lower the performance degradation, but the longer the time.
  :param logging: whether to log detailed information of model conversion, only valid when
                  accelerator='openvino', otherwise will be ignored. Default: ``True``.
  :param inplace: whether to perform inplace optimization. Default: ``False``.
  :param weights_prepack: Whether to perform weight prepack for convolution and linear to
                          avoid oneDNN weights reorder. The default value is None. Explicitly
                          setting this knob overwrites the configuration set by level knob.
                          Only valid when ``use_ipex=True``, otherwise will be ignored.
                          You can try to reduce the occupied memory size by setting this
                          parameter to ``False``.
  :param enable_onednn: Whether to use PyTorch JIT graph fuser based on oneDNN Graph API,
                        which provides a flexible API for aggressive fusion. Default to
                        ``False``, only valid when accelerator='jit', otherwise will
                        be ignored. For more details, please refer https://github.com/
                        pytorch/pytorch/tree/master/torch/csrc/jit/codegen/
                        onednn#pytorch---onednn-graph-api-bridge.
  :param q_config: Qconfig (https://pytorch.org/docs/stable/generated/torch.quantization.
                   qconfig.QConfig.html#qconfig) describes how to quantize a layer or a part
                   of the network by providing settings (observer classes) for activations
                   and weights respectively. Note that QConfig needs to contain observer
                   classes (like MinMaxObserver) or a callable that returns instances on
                   invocation, not the concrete observer instances themselves. Quantization
                   preparation function will instantiate observers multiple times for each
                   of the layers.
                   This parameter only works for native ipex and jit quantization with int8
                   precision. When accelerator='jit', we also support and recommend to pass
                   a QConfigMapping instead of single Qconfig for customized quantization.
                   QConfigMapping (https://pytorch.org/docs/stable/generated/torch.ao.
                   quantization.qconfig_mapping.QConfigMapping.html#qconfigmapping) is a
                   collection of quantization configurations, user can set the qconfig for
                   each operator (torch op calls, functional calls, module calls) in the
                   model through qconfig_mapping.
  :param output_tensors: boolean, default to True and output of the model will be Tensors,
                         only valid when accelerator='onnxruntime' or accelerator='openvino',
                         otherwise will be ignored. If output_tensors=False, output of the
                         export model will be ndarray.
  :param example_kwarg_inputs: a pack of keyword arguments of example inputs that will be
                               passed to ``torch.jit.trace``. Default: ``None``. Either
                               this argument or ``input_sample`` should be specified. The
                               dict will be unpacking by the arguments name of the traced
                               function. Only valid when accelerator='jit' and torch>=2.0,
                               otherwise will be ignored.
  :param **kwargs: Other extra advanced settings include:
                   1. those be passed to ``torch.onnx.export`` function,
                   only valid when accelerator='onnxruntime'/'openvino',
                   otherwise will be ignored.
                   Possible arguments are: input_names, output_names, opset_version,
                   et al. For more details, please refer
                   https://pytorch.org/docs/stable/onnx.html#torch.onnx.export.
                   2. those be passed to ``model optimizer`` function of openvino,
                   only valid when accelerator='openvino',
                   otherwise will be ignored.
                   Possible arguments are: mean_values, layout, input, output, et al.
                   For more details about model optimizer, you can see mo --help .
                   If you want to quantize with openvino on VPUX device,
                   you must specify  ``mean_value`` for model optimizer function.
                   Here ``mean_value`` represents mean values to be used for the input image
                   per channel. Values to be provided in the (R,G,B) or [R,G,B] format.
                   Can be defined for desired input of the model, for example:
                   "--mean_values data[255,255,255],info[255,255,255]". The exact meaning
                   and order of channels depend on how the original model was trained.
  :return:            A accelerated torch.nn.Module if quantization is successful.
docstr_tok: ["Calibrate","a","torch.nn.Module","for","post-training","quantization",".",":","param","model",":","A","model","to","be","quantized",".","Model","type","should","be","an","instance","of","torch.nn.Module",".",":","param","precision",":","Global","precision","of","quantized","model",",","supported","type",":","'int8","'",",","'bf16","'",",","'fp16","'",",","defaults","to","'int8","'",".",":","param","accelerator",":","Use","accelerator","'None","'",",","'onnxruntime","'",",","'openvino","'",",","'jit","'",",","defaults","to","None",".","None","means","staying","in","pytorch",".",":","param","use_ipex",":","Whether","we","use","ipex","as","accelerator","for","inference",".","If","precision","!","=","bf16",",","it","will","be","ignored",".","Default",":","``","False","``",".",":","param","calib_data",":","Calibration","data","is","required","for","static","quantization",".","It","'s","also","used","as","validation","dataloader",".","calib_data","support","following","formats",":","|","1.","a","torch.utils.data.dataloader.DataLoader","object","for","training",".","|","|","2.","a","single","torch.Tensor","used","for","training",",","this","case","is","used","|","to","accept","single","sample","input","x",".","|","|","3.","a","tuple","of","torch.Tensor","which","used","for","training",",","this","case","is","|","used","to","accept","single","sample","input","(","x",",","y",")","or","(","x1",",","x2",")","et","al",".",":","param","calib_dataloader",":","A","torch.utils.data.dataloader.DataLoader","object","for","calibration",".","Required","for","static","quantization",".","It","'s","also","used","as","validation","dataloader",".","..","warning",":",":","``","calib_dataloader","``","will","be","deprecated","in","future","release",".","Please","use","``","calib_data","``","instead",".",":","param","eval_func",":","A","evaluation","function","which","only","accepts","model","as","input","and","return","evaluation","value",".","This","parameter","provides","a","higher","degree","of","freedom","than","using","eval_loader","and","metric",".","Default","to","None","meaning","no","performance","tuning",",","but","it","would","be","better","give","an","evaluation","function","to","get","better","quantization","performance",".",":","param","metric",":","A","torchmetrics.metric.Metric","object","for","evaluation",".",":","param","accuracy_criterion",":","Tolerable","accuracy","drop",",","defaults","to","None","meaning","no","accuracy","control",".","accuracy_criterion","=","{","'absolute':0.99",",","'higher_is_better","'",":","False","}","means","accuracy","loss","must","be","smaller","than","0.99",".","For","example",",","if","higher_is_better","is","True",",","then","this","requires","original","metric","value","subtract","current","metric","value","be","smaller","than","0.99",".","For","inc","1.x",",","this","value","must","be","set","to","[","0",",","1",")",",","for","inc","2.x",",","there","is","no","limit",".","accuracy_criterion","=","{","'relative':0.1",",","'higher_is_better","'",":","True","}","allows","relative","accuracy","loss",":","10","%",".",":","param","approach",":","'static","'","or","'dynamic","'",".","'static","'",":","post_training_static_quant",",","'dynamic","'",":","post_training_dynamic_quant",".","Default",":","'static","'",".","OpenVINO","supports","static","mode","only",".",":","param","method",":","Method","to","do","quantization",".","When","accelerator=None",",","supported","methods",":","'fx","'",",","'eager","'",",","'ipex","'",",","defaults","to","'fx","'",".","If","you","do","n't","use","ipex",",","suggest","using","'fx","'","which","executes","automatic","optimizations","like","fusion",".","For","more","information",",","please","refer","to","https",":","\/\/pytorch.org\/docs\/stable\/quantization.html","#","eager-mode-quantization",".","When","accelerator='onnxruntime","'",",","supported","methods",":","'qlinear","'",",","'integer","'",",","defaults","to","'qlinear","'",".","Suggest","'qlinear","'","for","lower","accuracy","drop","if","using","static","quantization",".","More","details","in","https",":","\/\/onnxruntime.ai\/docs\/performance\/quantization.html",".","This","argument","does","n't","take","effect","for","OpenVINO",",","do","n't","change","it","for","OpenVINO",".",":","param","conf",":","A","path","to","conf","yaml","file","for","quantization",".","Default",":","None",",","using","default","config",".",":","param","tuning_strategy",":","'bayesian","'",",","'basic","'",",","'mse","'",",","'sigopt","'",".","Default",":","'bayesian","'",".",":","param","timeout",":","Tuning","timeout","(","seconds",")",".","Default",":","None",",","which","means","early","stop",".","Combine","with","max_trials","field","to","decide","when","to","exit",".",":","param","max_trials",":","Max","tune","times",".","Default",":","None",",","which","means","no","tuning",".","Combine","with","timeout","field","to","decide","when","to","exit",".","``","timeout=0",",","max_trials=1","''","means","it","will","try","quantization","only","once","and","return","satisfying","best","model",".",":","param","input_sample",":","An","input","example","to","convert","pytorch","model","into","ONNX\/OpenVINO\/JIT",".",":","param","channels_last",":","Whether","use","channels","last","memory","format",",","i.e",".","NHWC","(","batch","size",",","height",",","width",",","channels",")",",","as","an","alternative","way","to","store","tensors","in","classic\/contiguous","NCHW","order",",","only","valid","when","precision='bf16","'",",","otherwise","will","be","ignored",".","This","setting","only","works","for","4-dim","Tensor",".","Default",":","``","False","``",".",":","param","thread_num",":","(","optional",")","An","int","represents","how","many","threads","(","cores",")","is","needed","for","inference",".","This","parameter","only","controls","the","usage","of","thread","number","in","later","inference","process","of","your","obtained","accelerated","model",".","In","other","words",",","the","process","of","model","conversion","wo","n't","be","restricted","by","this","parameter",".",":","param","device",":","(","optional",")","A","string","represents","the","device","of","the","inference",".","Default","to","'CPU","'",",","only","valid","when","accelerator='openvino","'",",","otherwise","will","be","ignored",".","'CPU","'",",","'GPU","'","and","'VPUX","'","are","supported","for","now",".",":","param","onnxruntime_session_options",":","The","session","option","for","onnxruntime",",","only","valid","when","accelerator='onnxruntime","'",",","otherwise","will","be","ignored",".",":","param","openvino_config",":","The","config","to","be","inputted","in","core.compile_model",".","Only","valid","when","accelerator='openvino","'",",","otherwise","will","be","ignored",".",":","param","simplification",":","whether","we","use","onnxsim","to","simplify","the","ONNX","model",",","only","valid","when","accelerator='onnxruntime","'",",","otherwise","will","be","ignored",".","If","this","option","is","set","to","True",",","new","dependency","'onnxsim","'","need","to","be","installed",".",":","param","jit_strict",":","Whether","recording","your","mutable","container","types",".","This","parameter","will","be","passed","to","``","torch.jit.trace","``",".","if","``","accelerator","!","=","'jit","'","``","or","``","jit_method='script","'","``",",","it","will","be","ignored",".","Default","to","True",".",":","param","jit_method",":","Whether","to","use","``","jit.trace","``","or","``","jit.script","``","to","convert","a","model","to","TorchScript",".","Accepted","values","are","``","'trace","'","``",",","``","'script","'","``",",","and","``","None","``",".","Default","to","be","``","None","``","meaning","the","try-except","logic","to","use","``","jit.trace","``","or","``","jit.script","``",".","If","``","accelerator","!","=","'jit","'","``",",","this","parameter","will","be","ignored",".",":","param","dynamic_axes",":","dict","or","boolean",",","default","to","True",".","By","default","the","exported","onnx","model","will","have","the","first","dim","of","each","Tensor","input","as","a","dynamic","batch_size",".","If","dynamic_axes=False",",","the","exported","model","will","have","the","shapes","of","all","input","and","output","tensors","set","to","exactly","match","those","given","in","input_sample",".","To","specify","axes","of","tensors","as","dynamic","(","i.e",".","known","only","at","run-time",")",",","set","dynamic_axes","to","a","dict","with","schema",":","|","KEY","(","str",")",":","an","input","or","output","name",".","Each","name","must","also","be","provided","|","in","input_names","or","output_names",".","|","|","VALUE","(","dict","or","list",")",":","If","a","dict",",","keys","are","axis","indices","and","values","|","are","axis","names",".","If","a","list",",","each","element","is","an","axis","index",".","If","accelerator","!","=","'openvino'\/'onnxruntime","'",",","it","will","be","ignored",".",":","param","sample_size",":","(","optional",")","a","int","represents","how","many","samples","will","be","used","for","Post-training","Optimization","Tools","(","POT",")","from","OpenVINO","toolkit",",","only","valid","for","accelerator='openvino","'",".","Default","to","100",".","The","larger","the","value",",","the","more","accurate","the","conversion",",","the","lower","the","performance","degradation",",","but","the","longer","the","time",".",":","param","logging",":","whether","to","log","detailed","information","of","model","conversion",",","only","valid","when","accelerator='openvino","'",",","otherwise","will","be","ignored",".","Default",":","``","True","``",".",":","param","inplace",":","whether","to","perform","inplace","optimization",".","Default",":","``","False","``",".",":","param","weights_prepack",":","Whether","to","perform","weight","prepack","for","convolution","and","linear","to","avoid","oneDNN","weights","reorder",".","The","default","value","is","None",".","Explicitly","setting","this","knob","overwrites","the","configuration","set","by","level","knob",".","Only","valid","when","``","use_ipex=True","``",",","otherwise","will","be","ignored",".","You","can","try","to","reduce","the","occupied","memory","size","by","setting","this","parameter","to","``","False","``",".",":","param","enable_onednn",":","Whether","to","use","PyTorch","JIT","graph","fuser","based","on","oneDNN","Graph","API",",","which","provides","a","flexible","API","for","aggressive","fusion",".","Default","to","``","False","``",",","only","valid","when","accelerator='jit","'",",","otherwise","will","be","ignored",".","For","more","details",",","please","refer","https",":","\/\/github.com\/","pytorch\/pytorch\/tree\/master\/torch\/csrc\/jit\/codegen\/","onednn","#","pytorch","--","-onednn-graph-api-bridge",".",":","param","q_config",":","Qconfig","(","https",":","\/\/pytorch.org\/docs\/stable\/generated\/torch.quantization",".","qconfig.QConfig.html","#","qconfig",")","describes","how","to","quantize","a","layer","or","a","part","of","the","network","by","providing","settings","(","observer","classes",")","for","activations","and","weights","respectively",".","Note","that","QConfig","needs","to","contain","observer","classes","(","like","MinMaxObserver",")","or","a","callable","that","returns","instances","on","invocation",",","not","the","concrete","observer","instances","themselves",".","Quantization","preparation","function","will","instantiate","observers","multiple","times","for","each","of","the","layers",".","This","parameter","only","works","for","native","ipex","and","jit","quantization","with","int8","precision",".","When","accelerator='jit","'",",","we","also","support","and","recommend","to","pass","a","QConfigMapping","instead","of","single","Qconfig","for","customized","quantization",".","QConfigMapping","(","https",":","\/\/pytorch.org\/docs\/stable\/generated\/torch.ao",".","quantization.qconfig_mapping.QConfigMapping.html","#","qconfigmapping",")","is","a","collection","of","quantization","configurations",",","user","can","set","the","qconfig","for","each","operator","(","torch","op","calls",",","functional","calls",",","module","calls",")","in","the","model","through","qconfig_mapping",".",":","param","output_tensors",":","boolean",",","default","to","True","and","output","of","the","model","will","be","Tensors",",","only","valid","when","accelerator='onnxruntime","'","or","accelerator='openvino","'",",","otherwise","will","be","ignored",".","If","output_tensors=False",",","output","of","the","export","model","will","be","ndarray",".",":","param","example_kwarg_inputs",":","a","pack","of","keyword","arguments","of","example","inputs","that","will","be","passed","to","``","torch.jit.trace","``",".","Default",":","``","None","``",".","Either","this","argument","or","``","input_sample","``","should","be","specified",".","The","dict","will","be","unpacking","by","the","arguments","name","of","the","traced","function",".","Only","valid","when","accelerator='jit","'","and","torch",">","=2.0",",","otherwise","will","be","ignored",".",":","param","*","*","kwargs",":","Other","extra","advanced","settings","include",":","1.","those","be","passed","to","``","torch.onnx.export","``","function",",","only","valid","when","accelerator='onnxruntime'\/'openvino","'",",","otherwise","will","be","ignored",".","Possible","arguments","are",":","input_names",",","output_names",",","opset_version",",","et","al",".","For","more","details",",","please","refer","https",":","\/\/pytorch.org\/docs\/stable\/onnx.html","#","torch.onnx.export",".","2.","those","be","passed","to","``","model","optimizer","``","function","of","openvino",",","only","valid","when","accelerator='openvino","'",",","otherwise","will","be","ignored",".","Possible","arguments","are",":","mean_values",",","layout",",","input",",","output",",","et","al",".","For","more","details","about","model","optimizer",",","you","can","see","mo","--","help",".","If","you","want","to","quantize","with","openvino","on","VPUX","device",",","you","must","specify","``","mean_value","``","for","model","optimizer","function",".","Here","``","mean_value","``","represents","mean","values","to","be","used","for","the","input","image","per","channel",".","Values","to","be","provided","in","the","(","R",",","G",",","B",")","or","[","R",",","G",",","B","]","format",".","Can","be","defined","for","desired","input","of","the","model",",","for","example",":","``","--","mean_values","data","[","255,255,255","]",",","info","[","255,255,255","]","''",".","The","exact","meaning","and","order","of","channels","depend","on","how","the","original","model","was","trained",".",":","return",":","A","accelerated","torch.nn.Module","if","quantization","is","successful","."]
code: def quantize(
      model: nn.Module,
      precision: str = "int8",
      accelerator: Optional[str] = None,
      use_ipex: bool = False,
      calib_data: Union[
          DataLoader, torch.Tensor, Tuple[torch.Tensor]
      ] = None,
      calib_dataloader: Union[DataLoader] = None,
      eval_func: Optional[Callable] = None,
      metric: Optional[Metric] = None,
      accuracy_criterion: Optional[dict] = None,
      approach: str = "static",
      method: Optional[str] = None,
      conf: Optional[str] = None,
      tuning_strategy: Optional[str] = None,
      timeout: Optional[int] = None,
      max_trials: Optional[int] = None,
      input_sample=None,
      channels_last: bool = False,
      thread_num: Optional[int] = None,
      device: Optional[str] = "CPU",
      onnxruntime_session_options=None,
      openvino_config=None,
      simplification: bool = True,
      jit_strict: bool = True,
      jit_method: Optional[str] = None,
      dynamic_axes: Union[bool, dict] = True,
      sample_size: int = 100,
      logging: bool = True,
      inplace: bool = False,
      weights_prepack: Optional[bool] = None,
      enable_onednn: bool = False,
      q_config=None,
      output_tensors: bool = True,
      example_kwarg_inputs=None,
      **kwargs,
  ):
      """
      Calibrate a torch.nn.Module for post-training quantization.
  
      :param model:           A model to be quantized. Model type should be an instance of
                              torch.nn.Module.
      :param precision:       Global precision of quantized model,
                              supported type: 'int8', 'bf16', 'fp16', defaults to 'int8'.
      :param accelerator:     Use accelerator 'None', 'onnxruntime', 'openvino', 'jit', defaults
                              to None. None means staying in pytorch.
      :param use_ipex:        Whether we use ipex as accelerator for inference.
                              If precision != bf16, it will be ignored. Default: ``False``.
      :param calib_data:      Calibration data is required for static quantization.
                              It's also used as validation dataloader.
                              calib_data support following formats:
  
                              | 1. a torch.utils.data.dataloader.DataLoader object for training.
                              |
                              | 2. a single torch.Tensor used for training, this case is used
                              | to accept single sample input x.
                              |
                              | 3. a tuple of torch.Tensor which used for training, this case is
                              | used to accept single sample input (x, y) or (x1, x2) et al.
      :param calib_dataloader:    A torch.utils.data.dataloader.DataLoader object for calibration.
                                  Required for static quantization.
                                  It's also used as validation dataloader.
  
             .. warning::
                ``calib_dataloader`` will be deprecated in future release.
  
                Please use ``calib_data`` instead.
      :param eval_func:       A evaluation function which only accepts model as input and return
                              evaluation value. This parameter provides a higher degree of
                              freedom than using eval_loader and metric. Default to None meaning
                              no performance tuning, but it would be better give an evaluation
                              function to get better quantization performance.
      :param metric:              A torchmetrics.metric.Metric object for evaluation.
      :param accuracy_criterion:  Tolerable accuracy drop, defaults to None meaning no
                                  accuracy control.
                                  accuracy_criterion = {'absolute':0.99, 'higher_is_better':False}
                                  means accuracy loss must be smaller than 0.99. For example, if
                                  higher_is_better is True, then this requires original metric
                                  value subtract current metric value be smaller than 0.99.
                                  For inc 1.x, this value must be set to [0, 1), for inc 2.x,
                                  there is no limit.
                                  accuracy_criterion = {'relative':0.1, 'higher_is_better':True}
                                  allows relative accuracy loss: 10%.
      :param approach:    'static' or 'dynamic'.
                          'static': post_training_static_quant,
                          'dynamic': post_training_dynamic_quant.
                          Default: 'static'. OpenVINO supports static mode only.
      :param method:          Method to do quantization. When accelerator=None, supported
          methods: 'fx', 'eager', 'ipex', defaults to 'fx'. If you don't use ipex, suggest using
          'fx' which executes automatic optimizations like fusion. For more information, please
          refer to https://pytorch.org/docs/stable/quantization.html#eager-mode-quantization.
          When accelerator='onnxruntime', supported methods: 'qlinear', 'integer', defaults
          to 'qlinear'. Suggest 'qlinear' for lower accuracy drop if using static quantization.
          More details in https://onnxruntime.ai/docs/performance/quantization.html.
          This argument doesn't take effect for OpenVINO, don't change it for OpenVINO.
      :param conf:        A path to conf yaml file for quantization.
                          Default: None, using default config.
      :param tuning_strategy:    'bayesian', 'basic', 'mse', 'sigopt'. Default: 'bayesian'.
      :param timeout:     Tuning timeout (seconds). Default: None,  which means early stop.
                          Combine with max_trials field to decide when to exit.
      :param max_trials:  Max tune times. Default: None, which means no tuning.
                          Combine with timeout field to decide when to exit.
                          "timeout=0, max_trials=1" means it will try quantization only once and
                          return satisfying best model.
      :param input_sample:      An input example to convert pytorch model into ONNX/OpenVINO/JIT.
      :param channels_last: Whether use channels last memory format, i.e. NHWC (batch size,
                            height, width, channels), as an alternative way to store tensors in
                            classic/contiguous NCHW order, only valid when precision='bf16',
                            otherwise will be ignored. This setting only works for 4-dim Tensor.
                            Default: ``False``.
      :param thread_num: (optional) An int represents how many threads(cores) is needed for
                         inference. This parameter only controls the usage of thread number in
                         later inference process of your obtained accelerated model. In other
                         words, the process of model conversion won't be restricted by this
                         parameter.
      :param device: (optional) A string represents the device of the inference. Default to 'CPU',
                      only valid when accelerator='openvino', otherwise will be ignored.
                      'CPU', 'GPU' and 'VPUX' are supported for now.
      :param onnxruntime_session_options: The session option for onnxruntime, only valid when
                                          accelerator='onnxruntime', otherwise will be ignored.
      :param openvino_config: The config to be inputted in core.compile_model. Only valid when
                              accelerator='openvino', otherwise will be ignored.
      :param simplification: whether we use onnxsim to simplify the ONNX model, only valid when
                             accelerator='onnxruntime', otherwise will be ignored. If this option
                             is set to True, new dependency 'onnxsim' need to be installed.
      :param jit_strict: Whether recording your mutable container types. This parameter will be
                         passed to ``torch.jit.trace``. if ``accelerator != 'jit'`` or
                         ``jit_method='script'``, it will be ignored. Default to True.
      :param jit_method: Whether to use ``jit.trace`` or ``jit.script`` to convert a model
                         to TorchScript. Accepted values are ``'trace'``, ``'script'``,
                         and ``None``. Default to be ``None`` meaning the try-except logic
                         to use ``jit.trace`` or ``jit.script``. If ``accelerator != 'jit'``,
                         this parameter will be ignored.
      :param dynamic_axes: dict or boolean, default to True. By default the exported onnx model
                           will have the first dim of each Tensor input as a dynamic batch_size.
                           If dynamic_axes=False, the exported model will have the shapes of all
                           input and output tensors set to exactly match those given in
                           input_sample. To specify axes of tensors as dynamic (i.e. known only
                           at run-time), set dynamic_axes to a dict with schema:
  
                           | KEY (str): an input or output name. Each name must also be provided
                           | in input_names or output_names.
                           |
                           | VALUE (dict or list): If a dict, keys are axis indices and values
                           | are axis names. If a list, each element is an axis index.
  
                           If accelerator != 'openvino'/'onnxruntime', it will be ignored.
      :param sample_size: (optional) a int represents how many samples will be used for
                          Post-training Optimization Tools (POT) from OpenVINO toolkit,
                          only valid for accelerator='openvino'. Default to 100.
                          The larger the value, the more accurate the conversion,
                          the lower the performance degradation, but the longer the time.
      :param logging: whether to log detailed information of model conversion, only valid when
                      accelerator='openvino', otherwise will be ignored. Default: ``True``.
      :param inplace: whether to perform inplace optimization. Default: ``False``.
      :param weights_prepack: Whether to perform weight prepack for convolution and linear to
                              avoid oneDNN weights reorder. The default value is None. Explicitly
                              setting this knob overwrites the configuration set by level knob.
                              Only valid when ``use_ipex=True``, otherwise will be ignored.
                              You can try to reduce the occupied memory size by setting this
                              parameter to ``False``.
      :param enable_onednn: Whether to use PyTorch JIT graph fuser based on oneDNN Graph API,
                            which provides a flexible API for aggressive fusion. Default to
                            ``False``, only valid when accelerator='jit', otherwise will
                            be ignored. For more details, please refer https://github.com/
                            pytorch/pytorch/tree/master/torch/csrc/jit/codegen/
                            onednn#pytorch---onednn-graph-api-bridge.
      :param q_config: Qconfig (https://pytorch.org/docs/stable/generated/torch.quantization.
                       qconfig.QConfig.html#qconfig) describes how to quantize a layer or a part
                       of the network by providing settings (observer classes) for activations
                       and weights respectively. Note that QConfig needs to contain observer
                       classes (like MinMaxObserver) or a callable that returns instances on
                       invocation, not the concrete observer instances themselves. Quantization
                       preparation function will instantiate observers multiple times for each
                       of the layers.
                       This parameter only works for native ipex and jit quantization with int8
                       precision. When accelerator='jit', we also support and recommend to pass
                       a QConfigMapping instead of single Qconfig for customized quantization.
                       QConfigMapping (https://pytorch.org/docs/stable/generated/torch.ao.
                       quantization.qconfig_mapping.QConfigMapping.html#qconfigmapping) is a
                       collection of quantization configurations, user can set the qconfig for
                       each operator (torch op calls, functional calls, module calls) in the
                       model through qconfig_mapping.
      :param output_tensors: boolean, default to True and output of the model will be Tensors,
                             only valid when accelerator='onnxruntime' or accelerator='openvino',
                             otherwise will be ignored. If output_tensors=False, output of the
                             export model will be ndarray.
      :param example_kwarg_inputs: a pack of keyword arguments of example inputs that will be
                                   passed to ``torch.jit.trace``. Default: ``None``. Either
                                   this argument or ``input_sample`` should be specified. The
                                   dict will be unpacking by the arguments name of the traced
                                   function. Only valid when accelerator='jit' and torch>=2.0,
                                   otherwise will be ignored.
      :param **kwargs: Other extra advanced settings include:
                       1. those be passed to ``torch.onnx.export`` function,
                       only valid when accelerator='onnxruntime'/'openvino',
                       otherwise will be ignored.
                       Possible arguments are: input_names, output_names, opset_version,
                       et al. For more details, please refer
                       https://pytorch.org/docs/stable/onnx.html#torch.onnx.export.
                       2. those be passed to ``model optimizer`` function of openvino,
                       only valid when accelerator='openvino',
                       otherwise will be ignored.
                       Possible arguments are: mean_values, layout, input, output, et al.
                       For more details about model optimizer, you can see mo --help .
                       If you want to quantize with openvino on VPUX device,
                       you must specify  ``mean_value`` for model optimizer function.
                       Here ``mean_value`` represents mean values to be used for the input image
                       per channel. Values to be provided in the (R,G,B) or [R,G,B] format.
                       Can be defined for desired input of the model, for example:
                       "--mean_values data[255,255,255],info[255,255,255]". The exact meaning
                       and order of channels depend on how the original model was trained.
      :return:            A accelerated torch.nn.Module if quantization is successful.
      """
      invalidInputError(
          precision in ["int8", "fp16", "bf16"],
          "Only support 'int8', 'bf16', 'fp16' now, "
          "no support for {}.".format(precision),
      )
      # device name might be: CPU, GPU, GPU.0, VPUX ...
      invalidInputError(
          device == "CPU" or "GPU" in device or device == "VPUX",
          "Now we only support CPU, GPU and VPUX, not {}".format(
              device
          ),
      )
      if device != "CPU" and accelerator not in ["openvino", None]:
          invalidInputError(
              False,
              "Now we only support {} device when accelerator"
              "is openvino.".format(device),
          )
      model.eval()  # change model to eval mode
      if precision == "bf16":
          if accelerator is None or accelerator == "jit":
              if use_ipex or accelerator == "jit":
                  if use_ipex is True:
                      invalidInputError(
                          not TORCH_VERSION_LESS_1_10,
                          "torch version should >=1.10 to use ipex",
                      )
                  use_jit = accelerator == "jit"
                  if use_jit:
                      invalidInputError(
                          jit_method in [None, "trace", "script"],
                          "jit_method {} is invalid.".format(
                              jit_method
                          ),
                      )
                  return PytorchIPEXJITBF16Model(
                      model,
                      input_sample=input_sample,
                      use_ipex=use_ipex,
                      use_jit=use_jit,
                      channels_last=channels_last,
                      thread_num=thread_num,
                      inplace=inplace,
                      jit_strict=jit_strict,
                      jit_method=jit_method,
                      weights_prepack=weights_prepack,
                      enable_onednn=enable_onednn,
                      example_kwarg_inputs=example_kwarg_inputs,
                  )
              else:
                  bf16_model = BF16Model(
                      model,
                      channels_last=channels_last,
                      input_sample=input_sample,
                      thread_num=thread_num,
                  )
                  return bf16_model
          elif accelerator == "openvino":
              invalidInputError(
                  device == "CPU",
                  "Device {} don't support bfloat16.".format(device),
              )
              final_openvino_option = {
                  "INFERENCE_PRECISION_HINT": "bf16"
              }
              if openvino_config is not None:
                  final_openvino_option.update(openvino_config)
              return PytorchOpenVINOModel(
                  model,
                  input_sample,
                  thread_num=thread_num,
                  device=device,
                  dynamic_axes=dynamic_axes,
                  logging=logging,
                  config=final_openvino_option,
                  output_tensors=output_tensors,
                  **kwargs,
              )
          else:
              invalidInputError(
                  False,
                  "Accelerator {} is invalid for BF16.".format(
                      accelerator
                  ),
              )
      if precision == "int8":
          # transform non-dataloader to dataloader
          if calib_data is not None and not isinstance(
              calib_data, DataLoader
          ):
              dataset = RepeatDataset(sample=calib_data, num=1)
              calib_dataloader = DataLoader(dataset, batch_size=1)
              calib_dataloader = remove_batch_dim_fn(calib_dataloader)
          else:
              if calib_data is None and calib_dataloader is not None:
                  # will be deprecated in future release
                  warnings.warn(
                      "`calib_dataloader` will be deprecated in future release, please"
                      "use `calib_data` instead.",
                      category=DeprecationWarning,
                  )
                  calib_dataloader = calib_dataloader
              else:
                  calib_dataloader = calib_data
          # judge whether contains label in calib_dataloader
          # if not, will append label at last
          if accelerator is not None:
              calib_dataloader = automatic_add_label_in_dataloader(
                  model, calib_dataloader, input_sample
              )
  
          # transform the dataloader to inc mode
          inc_calib_dataloader = (
              transform_multiple_input_dataloader_to_inc_mode(
                  model, calib_dataloader
              )
          )
  
          if not accelerator or accelerator == "onnxruntime":
              method_map = {
                  None: {
                      "fx": "pytorch_fx",
                      "eager": "pytorch",
                      "ipex": "pytorch_ipex",
                      None: "pytorch_fx",  # default
                  },
                  "onnxruntime": {
                      "qlinear": "onnxrt_qlinearops",
                      "integer": "onnxrt_integerops",
                      None: "onnxrt_qlinearops",  # default
                  },
              }
              framework = method_map[accelerator].get(method, None)
              if accelerator == "onnxruntime":
                  if (
                      not type(model).__name__
                      == "PytorchONNXRuntimeModel"
                  ):
                      # try to establish onnx model
                      if input_sample is None:
                          # input_sample can be a dataloader
                          input_sample = calib_dataloader
                      model = InferenceOptimizer.trace(
                          model,
                          input_sample=input_sample,
                          accelerator="onnxruntime",
                          thread_num=thread_num,
                          onnxruntime_session_options=onnxruntime_session_options,
                          simplification=simplification,
                          dynamic_axes=dynamic_axes,
                          output_tensors=output_tensors,
                          **kwargs,
                      )
              """
                  If accelerator==None, quantized model returned should be an object of PytorchModel
                  which is defined by neural-compressor containing a `GraphModule` for inference.
                  Otherwise accelerator=='onnxruntime', it returns an ONNXModel object. A supported
                  model which is able to run on Pytorch or ONNXRuntime can be fetched by
                  `quantized_model.model`.
                  """
              inc_quantize_arguments = {
                  "model": model,
                  "dataloader": inc_calib_dataloader,
                  "eval_func": eval_func,
                  "metric": metric,
                  "thread_num": thread_num,
                  "framework": framework,
                  "conf": conf,
                  "approach": approach,
                  "tuning_strategy": tuning_strategy,
                  "accuracy_criterion": accuracy_criterion,
                  "timeout": timeout,
                  "max_trials": max_trials,
                  "onnxruntime_session_options": onnxruntime_session_options,
              }
              if accelerator == "onnxruntime":
                  q_model = inc_quantize(**inc_quantize_arguments)
                  q_model.output_tensors = output_tensors
                  return q_model
              elif framework != "pytorch_ipex":
                  return inc_quantize(**inc_quantize_arguments)
              else:
                  try:
                      return inc_quantize(**inc_quantize_arguments)
                  except Exception:
                      # use pure ipex quantization as a backup for inc ipex quantization
                      return PytorchIPEXQuantizationModel(
                          model,
                          inc_calib_dataloader,
                          q_config=q_config,
                          input_sample=input_sample,
                          channels_last=channels_last,
                          thread_num=thread_num,
                          inplace=inplace,
                          jit_strict=jit_strict,
                          enable_onednn=enable_onednn,
                      )
          elif accelerator == "openvino":
              model_type = type(model).__name__
              if not model_type == "PytorchOpenVINOModel":
                  if input_sample is None:
                      # input_sample can be a dataloader
                      input_sample = calib_dataloader
                  # For CPU: fp32 -> int8, for GPU/VPUX: fp16 -> int8
                  _precision = "fp16" if device != "CPU" else "fp32"
                  if device == "VPUX":
                      # for fp16 on VPUX, must specify mean_value.
                      invalidInputError(
                          "mean_value" in kwargs,
                          "If you want to quantize with openvino on VPUX device, "
                          "you must specify mean_value for model optimizer "
                          "function. For more details about model optimizer, you "
                          "can see mo --help .",
                      )
                  model = PytorchOpenVINOModel(
                      model,
                      input_sample,
                      precision=_precision,
                      thread_num=thread_num,
                      device=device,
                      dynamic_axes=dynamic_axes,
                      logging=logging,
                      config=openvino_config,
                      output_tensors=output_tensors,
                      **kwargs,
                  )
              invalidInputError(
                  type(model).__name__ == "PytorchOpenVINOModel",
                  "Invalid model to quantize. Please use a nn.Module or a model "
                  "from InferenceOptimizer.trace(accelerator=='openvino')",
              )
              drop_type = None
              higher_is_better = None
              maximal_drop = None
              if metric:
                  if not isinstance(accuracy_criterion, dict):
                      accuracy_criterion = {
                          "relative": 0.99,
                          "higher_is_better": True,
                      }
  
                  drop_type = (
                      "relative"
                      if "relative" in accuracy_criterion
                      else "absolute"
                  )
                  higher_is_better = accuracy_criterion.get(
                      "higher_is_better", None
                  )
                  maximal_drop = accuracy_criterion.get(drop_type, None)
  
              kwargs = {
                  "metric": metric,
                  "higher_better": higher_is_better,
                  "drop_type": drop_type,
                  "maximal_drop": maximal_drop,
                  "max_iter_num": max_trials,
                  # TODO following two keys are optional, if there is need, we can add them
                  # "n_requests": None,
                  "sample_size": sample_size,
              }
              return model.pot(
                  calib_dataloader,
                  thread_num=thread_num,
                  config=openvino_config,
                  **kwargs,
              )
          elif accelerator == "jit":
              invalidInputError(
                  jit_method in [None, "trace", "script"],
                  "jit_method {} is invalid.".format(jit_method),
              )
              return PytorchJITINT8Model(
                  model,
                  calib_dataloader,
                  q_config=q_config,
                  input_sample=input_sample,
                  channels_last=channels_last,
                  thread_num=thread_num,
                  jit_strict=jit_strict,
                  jit_method=jit_method,
                  enable_onednn=enable_onednn,
                  example_kwarg_inputs=example_kwarg_inputs,
              )
          else:
              invalidInputError(
                  False,
                  "Accelerator {} is invalid.".format(accelerator),
              )
      if precision == "fp16":
          invalidInputError(
              accelerator in ["openvino", None],
              "fp16 is not supported on {} accelerator.".format(
                  accelerator
              ),
          )
          if device == "VPUX":
              # for fp16 on VPUX, must specify mean_value.
              invalidInputError(
                  "mean_value" in kwargs,
                  "If you want to quantize with openvino float16 precision on "
                  "VPUX device, you must specify mean_value for model optimizer "
                  "function. For more details about model optimizer, you can "
                  "see mo --help .",
              )
          if accelerator is None and device == "GPU":
              return PytorchIPEXPUModel(
                  model,
                  thread_num=thread_num,
                  precision=precision,
                  use_ipex=use_ipex,
              ).to("xpu")
          return PytorchOpenVINOModel(
              model,
              input_sample,
              precision=precision,
              thread_num=thread_num,
              device=device,
              dynamic_axes=dynamic_axes,
              logging=logging,
              config=openvino_config,
              output_tensors=output_tensors,
              **kwargs,
          )
  
      invalidInputError(
          False, "Precision {} is invalid.".format(precision)
      )
  
imports: import torch
  from torch import nn
  import time
  import sigfig
  import multiprocessing
  from typing import Dict, Callable, Tuple, Optional, List, Union, Sequence, Mapping
  from torch.utils.data import DataLoader
  from torchmetrics.metric import Metric
  from bigdl.nano.utils.common import AccelerationOption, available_acceleration_combination, latency_calculate_helper, torch_loader_latency_calculate_helper, format_optimize_result, BaseInferenceOptimizer
  from bigdl.nano.utils.common import invalidInputError
  from bigdl.nano.pytorch.amp import BF16Model
  from bigdl.nano.pytorch.low_precision.jit_int8_api import PytorchJITINT8Model
  from bigdl.nano.deps.openvino.openvino_api import PytorchOpenVINOModel
  from bigdl.nano.deps.ipex.ipex_api import PytorchIPEXJITModel, PytorchIPEXJITBF16Model, PytorchIPEXQuantizationModel, PytorchIPEXPUModel
  from bigdl.nano.deps.onnxruntime.onnxruntime_api import PytorchONNXRuntimeModel
  from bigdl.nano.deps.neural_compressor.inc_api import quantize
  from bigdl.nano.pytorch.model import AcceleratedLightningModule
  from bigdl.nano.utils.pytorch import get_forward_args, get_input_example
  from bigdl.nano.utils.pytorch import NanoMetric
  from bigdl.nano.utils.pytorch import RepeatDataset, remove_batch_dim_fn
  from bigdl.nano.utils.pytorch import transform_multiple_input_dataloader_to_inc_mode, automatic_add_label_in_dataloader
  from bigdl.nano.utils.pytorch import TORCH_VERSION_LESS_1_10
  from bigdl.nano.utils.pytorch import save_model, load_model
  from bigdl.nano.utils.common import schedule_processors
  from bigdl.nano.utils.common import EnvContext
  from bigdl.nano.pytorch.context_manager import generate_context_manager, BaseContextManager, AutocastContextManager
  from .multi_instance import _MultiInstanceModel, _multi_instance_helper
  import traceback
  import warnings
  import os
label_desc: Use image node_id 2 for calling the InferenceOptimizer obj's underlying member method code with example usage: obj.quantize(model, precision, accelerator, use_ipex, calib_data, calib_dataloader, eval_func, metric, accuracy_criterion, approach, method, conf, tuning_strategy, timeout, max_trials, input_sample, channels_last, thread_num, device, onnxruntime_session_options, openvino_config, simplification, jit_strict, jit_method, dynamic_axes, sample_size, logging, inplace, weights_prepack, enable_onednn, q_config, output_tensors, example_kwarg_inputs) and returns: PytorchOpenVINOModel, PytorchIPEXPUModel, PytorchIPEXJITBF16Model, bf16_model, PytorchOpenVINOModel, q_model, model, inc_quantize, PytorchJITINT8Model, inc_quantize, PytorchIPEXQuantizationModel