docstr: This function will give all available inference acceleration methods a try
  and record the latency, accuracy and model instance inside the Optimizer for
  future usage. All model instance is setting to eval mode.
  
  The available methods are "original", "fp32_channels_last", "fp32_ipex",
  "fp32_ipex_channels_last", "bf16", "bf16_channels_last", "bf16_ipex",
  "bf16_ipex_channels_last", "static_int8", "static_int8_ipex", "jit_fp32",
  "jit_fp32_channels_last", "jit_bf16", "jit_bf16_channels_last",
  "jit_fp32_ipex", "jit_fp32_ipex_channels_last", "jit_bf16_ipex",
  "jit_bf16_ipex_channels_last", "jit_int8", "jit_int8_channels_last",
  "openvino_fp32", "openvino_int8", "onnxruntime_fp32",
  "onnxruntime_int8_qlinear" and "onnxruntime_int8_integer".
  
  :param model: A torch.nn.Module to be optimized
  :param training_data: training_data support following formats:
  
          | 1. a torch.utils.data.dataloader.DataLoader object for training dataset.
          | Users should be careful with this parameter since this dataloader
          | might be exposed to the model, which causing data leak. The
          | batch_size of this dataloader is important as well, users may
          | want to set it to the same batch size you may want to use the model
          | in real deploy environment. E.g. batch size should be set to 1
          | if you would like to use the accelerated model in an online service.
          |
          | Each element in the DataLoader can be one of the following:
          |    a. a single Tensor or dict of Tensors
          |    b. a tuple:
          |         b1: if the length is 1, the first element will be treated as input
          |             to the model
          |         b2: if the length is 2, the first element will be treated as input
          |             to the model, with the sencond element treated as label.
          |             if the input to the model is a tuple, it will be unpacked as
          |             multiple inputs.
          |         b3: if the length is larger than 2, the first n elements as input
          |             to the model, with n being the argument lenth to the model.forward
          |             and the rest will be treated as label
          |
          | 2. a single element of the Dataloader specified above
  
  :param validation_data: (optional) validation_data is only needed when users care
                          about the possible accuracy drop. It support following formats:
  
          | 1. a torch.utils.data.dataloader.DataLoader object for accuracy evaluation.
          |
          | Each element in the DataLoader should be a tuple as least size of two:
          |     a: if the length is 2, the first element will be treated as input
          |        to the model, with the sencond element treated as label
          |     b: if the length is larger than 2, the first n elements as input
          |        to the model, with n being the argument lenth to the model.forward
          |        and the rest will be treated as label
          |
          | 2. a single element of the Dataloader specified above
  
  :param input_sample: (optional) A set of inputs for trace, defaults to None.
         In most cases, you don't need specify this parameter, it will be obtained from
         training_data. You have to specify this parameter only if the forward function
         of your model contains some kwargs like `def forward(self, x1, x2, x3=1)`.
  :param metric: (optional) A callable object which is used for calculating accuracy.
         It supports two kinds of callable object:
  
         | 1. A torchmetrics.Metric object or similar callable object which takes
         | prediction and target then returns an accuracy value in this calling
         | method `metric(pred, target)`. This requires data in validation_data
         | is composed of (input_data, target).
         |
         | 2. A callable object that takes model and validation_data (if
         | validation_data is not None) as input, and returns an accuracy value in
         | this calling method metric(model, data_loader) (or metric(model) if
         | validation_data is None). Note that there is no need to call `with
         | InferenceOptimizer.get_context()` in this object.
  
  :param direction: (optional) A string that indicates the higher/lower
         better for the metric, "min" for the lower the better and "max" for the
         higher the better. Default value is "max".
  :param thread_num: (optional) An int represents how many threads(cores) is needed for
         inference. This parameter only controls the usage of thread number in the process
         of latency calculation as well as later inference process of your obtained
         accelerated model. In other words, the process of model conversion and optional
         accuracy calculation won't be restricted by this parameter. Defaults to None,
         represents that all cores will be used.
  :param accelerator: (optional) A string tuple that specifies the accelerators to search.
         The optional accelerators are: None, 'openvino', 'onnxruntime', 'jit'.
         Defaults to None which represents there is no restriction on accelerators.
         If not None, then will only traverse corresponding methods whose accelerator falls
         within the specified accelerator tuple.
  :param precision: (optional) A string tuple that specifies the precision to search.
         The optional precision are: 'int8', 'bf16', and 'fp32'. Defaults to None which
         represents no precision limit. If not None, then will only traverse corresponding
         methods whose precision falls within the specified precision tuple.
  :param use_ipex: (optional) if not None, then will only try methods with/without
         this specific ipex setting.
  :param jit_strict: Whether recording your mutable container types. This parameter will be
         passed to ``torch.jit.trace``. if ``accelerator != 'jit'`` or
         ``jit_method='script'``, it will be ignored. Default to True.
  :param enable_onednn: Whether to use PyTorch JIT graph fuser based on oneDNN Graph API,
          which provides a flexible API for aggressive fusion. Default to
          ``False``, only valid when accelerator='jit', otherwise will
          be ignored. For more details, please refer https://github.com/
          pytorch/pytorch/tree/master/torch/csrc/jit/codegen/
          onednn#pytorch---onednn-graph-api-bridge.
  :param search_mode: Here are three modes for optimization:
  
         | 1. default: This mode only traverses a subset of all combinations. This subset
         | is a collection of methods that we select based on experience and think have
         | better acceleration effect in general. This mode allows you to quickly obtain a
         | good acceleration method, but it is not necessarily the global optimal. Default
         | to this mode if you don't specify accelerator/precision/use_ipex.
         |
         | 2. all: This mode will traverse all possible combinations, which can ensure
         | find the global optimization, but it will take a long time.
         |
         | 3. grid: If you have specified accelerator/precision/use_ipex, the default is
         | grid mode. We will sort and combine according to the value you specified to
         | get the search range.
  
  :param dynamic_axes: dict or boolean, default to True. By default the exported onnx model
         will have the first dim of each Tensor input as a dynamic batch_size. If
         dynamic_axes=False, the exported model will have the shapes of all input and output
         tensors set to exactly match those given in input_sample. To specify axes of
         tensors as dynamic (i.e. known only at run-time), set dynamic_axes to a dict with
         schema:
  
         | KEY (str): an input or output name. Each name must also be provided
         | in input_names or output_names.
         |
         | VALUE (dict or list): If a dict, keys are axis indices and values are
         | axis names. If a list, each element is an axis index.
  
         If accelerator != 'openvino'/'onnxruntime', it will be ignored.
  :param logging: whether to log detailed information of model conversion.
         Default: False.
  :param output_tensors: boolean, default to True and output of the model will be Tensors,
                         only valid when accelerator='onnxruntime' or accelerator='openvino',
                         otherwise will be ignored. If output_tensors=False, output of the
                         export model will be ndarray.
  :param latency_sample_num: (optional) a int represents the number of repetitions
         to calculate the average latency. The default value is 100.
  :param includes: (optional) a list of acceleration methods that will be included in the
         search. Default to None meaning including all available methods. "original" method
         will be automatically add to includes.
  :param excludes: (optional) a list of acceleration methods that will be excluded from the
         search. "original" will be ignored in the excludes.
  :param output_filename: (optional) a string filename is used to specify the file which the
         optimized table will be writed. The default is None which means don't write to file.
  :param no_cache: if set True, calculate average latency by iterating all the samples from
         the provided dataloader until reaching the latency_sample_num. Default set to be
         False, meaning always loading one single sample from cache to test latency.
docstr_tok: ["This","function","will","give","all","available","inference","acceleration","methods","a","try","and","record","the","latency",",","accuracy","and","model","instance","inside","the","Optimizer","for","future","usage",".","All","model","instance","is","setting","to","eval","mode",".","The","available","methods","are","``","original","''",",","``","fp32_channels_last","''",",","``","fp32_ipex","''",",","''","fp32_ipex_channels_last","''",",","``","bf16","''",",","``","bf16_channels_last","''",",","``","bf16_ipex","''",",","''","bf16_ipex_channels_last","''",",","``","static_int8","''",",","``","static_int8_ipex","''",",","``","jit_fp32","''",",","''","jit_fp32_channels_last","''",",","``","jit_bf16","''",",","``","jit_bf16_channels_last","''",",","''","jit_fp32_ipex","''",",","``","jit_fp32_ipex_channels_last","''",",","``","jit_bf16_ipex","''",",","''","jit_bf16_ipex_channels_last","''",",","``","jit_int8","''",",","``","jit_int8_channels_last","''",",","''","openvino_fp32","''",",","``","openvino_int8","''",",","``","onnxruntime_fp32","''",",","''","onnxruntime_int8_qlinear","''","and","``","onnxruntime_int8_integer","''",".",":","param","model",":","A","torch.nn.Module","to","be","optimized",":","param","training_data",":","training_data","support","following","formats",":","|","1.","a","torch.utils.data.dataloader.DataLoader","object","for","training","dataset",".","|","Users","should","be","careful","with","this","parameter","since","this","dataloader","|","might","be","exposed","to","the","model",",","which","causing","data","leak",".","The","|","batch_size","of","this","dataloader","is","important","as","well",",","users","may","|","want","to","set","it","to","the","same","batch","size","you","may","want","to","use","the","model","|","in","real","deploy","environment",".","E.g",".","batch","size","should","be","set","to","1","|","if","you","would","like","to","use","the","accelerated","model","in","an","online","service",".","|","|","Each","element","in","the","DataLoader","can","be","one","of","the","following",":","|","a.","a","single","Tensor","or","dict","of","Tensors","|","b.","a","tuple",":","|","b1",":","if","the","length","is","1",",","the","first","element","will","be","treated","as","input","|","to","the","model","|","b2",":","if","the","length","is","2",",","the","first","element","will","be","treated","as","input","|","to","the","model",",","with","the","sencond","element","treated","as","label",".","|","if","the","input","to","the","model","is","a","tuple",",","it","will","be","unpacked","as","|","multiple","inputs",".","|","b3",":","if","the","length","is","larger","than","2",",","the","first","n","elements","as","input","|","to","the","model",",","with","n","being","the","argument","lenth","to","the","model.forward","|","and","the","rest","will","be","treated","as","label","|","|","2.","a","single","element","of","the","Dataloader","specified","above",":","param","validation_data",":","(","optional",")","validation_data","is","only","needed","when","users","care","about","the","possible","accuracy","drop",".","It","support","following","formats",":","|","1.","a","torch.utils.data.dataloader.DataLoader","object","for","accuracy","evaluation",".","|","|","Each","element","in","the","DataLoader","should","be","a","tuple","as","least","size","of","two",":","|","a",":","if","the","length","is","2",",","the","first","element","will","be","treated","as","input","|","to","the","model",",","with","the","sencond","element","treated","as","label","|","b",":","if","the","length","is","larger","than","2",",","the","first","n","elements","as","input","|","to","the","model",",","with","n","being","the","argument","lenth","to","the","model.forward","|","and","the","rest","will","be","treated","as","label","|","|","2.","a","single","element","of","the","Dataloader","specified","above",":","param","input_sample",":","(","optional",")","A","set","of","inputs","for","trace",",","defaults","to","None",".","In","most","cases",",","you","do","n't","need","specify","this","parameter",",","it","will","be","obtained","from","training_data",".","You","have","to","specify","this","parameter","only","if","the","forward","function","of","your","model","contains","some","kwargs","like","`","def","forward","(","self",",","x1",",","x2",",","x3=1",")","`",".",":","param","metric",":","(","optional",")","A","callable","object","which","is","used","for","calculating","accuracy",".","It","supports","two","kinds","of","callable","object",":","|","1",".","A","torchmetrics.Metric","object","or","similar","callable","object","which","takes","|","prediction","and","target","then","returns","an","accuracy","value","in","this","calling","|","method","`","metric","(","pred",",","target",")","`",".","This","requires","data","in","validation_data","|","is","composed","of","(","input_data",",","target",")",".","|","|","2",".","A","callable","object","that","takes","model","and","validation_data","(","if","|","validation_data","is","not","None",")","as","input",",","and","returns","an","accuracy","value","in","|","this","calling","method","metric","(","model",",","data_loader",")","(","or","metric","(","model",")","if","|","validation_data","is","None",")",".","Note","that","there","is","no","need","to","call","`","with","|","InferenceOptimizer.get_context","(",")","`","in","this","object",".",":","param","direction",":","(","optional",")","A","string","that","indicates","the","higher\/lower","better","for","the","metric",",","``","min","''","for","the","lower","the","better","and","``","max","''","for","the","higher","the","better",".","Default","value","is","``","max","''",".",":","param","thread_num",":","(","optional",")","An","int","represents","how","many","threads","(","cores",")","is","needed","for","inference",".","This","parameter","only","controls","the","usage","of","thread","number","in","the","process","of","latency","calculation","as","well","as","later","inference","process","of","your","obtained","accelerated","model",".","In","other","words",",","the","process","of","model","conversion","and","optional","accuracy","calculation","wo","n't","be","restricted","by","this","parameter",".","Defaults","to","None",",","represents","that","all","cores","will","be","used",".",":","param","accelerator",":","(","optional",")","A","string","tuple","that","specifies","the","accelerators","to","search",".","The","optional","accelerators","are",":","None",",","'openvino","'",",","'onnxruntime","'",",","'jit","'",".","Defaults","to","None","which","represents","there","is","no","restriction","on","accelerators",".","If","not","None",",","then","will","only","traverse","corresponding","methods","whose","accelerator","falls","within","the","specified","accelerator","tuple",".",":","param","precision",":","(","optional",")","A","string","tuple","that","specifies","the","precision","to","search",".","The","optional","precision","are",":","'int8","'",",","'bf16","'",",","and","'fp32","'",".","Defaults","to","None","which","represents","no","precision","limit",".","If","not","None",",","then","will","only","traverse","corresponding","methods","whose","precision","falls","within","the","specified","precision","tuple",".",":","param","use_ipex",":","(","optional",")","if","not","None",",","then","will","only","try","methods","with\/without","this","specific","ipex","setting",".",":","param","jit_strict",":","Whether","recording","your","mutable","container","types",".","This","parameter","will","be","passed","to","``","torch.jit.trace","``",".","if","``","accelerator","!","=","'jit","'","``","or","``","jit_method='script","'","``",",","it","will","be","ignored",".","Default","to","True",".",":","param","enable_onednn",":","Whether","to","use","PyTorch","JIT","graph","fuser","based","on","oneDNN","Graph","API",",","which","provides","a","flexible","API","for","aggressive","fusion",".","Default","to","``","False","``",",","only","valid","when","accelerator='jit","'",",","otherwise","will","be","ignored",".","For","more","details",",","please","refer","https",":","\/\/github.com\/","pytorch\/pytorch\/tree\/master\/torch\/csrc\/jit\/codegen\/","onednn","#","pytorch","--","-onednn-graph-api-bridge",".",":","param","search_mode",":","Here","are","three","modes","for","optimization",":","|","1.","default",":","This","mode","only","traverses","a","subset","of","all","combinations",".","This","subset","|","is","a","collection","of","methods","that","we","select","based","on","experience","and","think","have","|","better","acceleration","effect","in","general",".","This","mode","allows","you","to","quickly","obtain","a","|","good","acceleration","method",",","but","it","is","not","necessarily","the","global","optimal",".","Default","|","to","this","mode","if","you","do","n't","specify","accelerator\/precision\/use_ipex",".","|","|","2.","all",":","This","mode","will","traverse","all","possible","combinations",",","which","can","ensure","|","find","the","global","optimization",",","but","it","will","take","a","long","time",".","|","|","3.","grid",":","If","you","have","specified","accelerator\/precision\/use_ipex",",","the","default","is","|","grid","mode",".","We","will","sort","and","combine","according","to","the","value","you","specified","to","|","get","the","search","range",".",":","param","dynamic_axes",":","dict","or","boolean",",","default","to","True",".","By","default","the","exported","onnx","model","will","have","the","first","dim","of","each","Tensor","input","as","a","dynamic","batch_size",".","If","dynamic_axes=False",",","the","exported","model","will","have","the","shapes","of","all","input","and","output","tensors","set","to","exactly","match","those","given","in","input_sample",".","To","specify","axes","of","tensors","as","dynamic","(","i.e",".","known","only","at","run-time",")",",","set","dynamic_axes","to","a","dict","with","schema",":","|","KEY","(","str",")",":","an","input","or","output","name",".","Each","name","must","also","be","provided","|","in","input_names","or","output_names",".","|","|","VALUE","(","dict","or","list",")",":","If","a","dict",",","keys","are","axis","indices","and","values","are","|","axis","names",".","If","a","list",",","each","element","is","an","axis","index",".","If","accelerator","!","=","'openvino'\/'onnxruntime","'",",","it","will","be","ignored",".",":","param","logging",":","whether","to","log","detailed","information","of","model","conversion",".","Default",":","False",".",":","param","output_tensors",":","boolean",",","default","to","True","and","output","of","the","model","will","be","Tensors",",","only","valid","when","accelerator='onnxruntime","'","or","accelerator='openvino","'",",","otherwise","will","be","ignored",".","If","output_tensors=False",",","output","of","the","export","model","will","be","ndarray",".",":","param","latency_sample_num",":","(","optional",")","a","int","represents","the","number","of","repetitions","to","calculate","the","average","latency",".","The","default","value","is","100.",":","param","includes",":","(","optional",")","a","list","of","acceleration","methods","that","will","be","included","in","the","search",".","Default","to","None","meaning","including","all","available","methods",".","``","original","''","method","will","be","automatically","add","to","includes",".",":","param","excludes",":","(","optional",")","a","list","of","acceleration","methods","that","will","be","excluded","from","the","search",".","``","original","''","will","be","ignored","in","the","excludes",".",":","param","output_filename",":","(","optional",")","a","string","filename","is","used","to","specify","the","file","which","the","optimized","table","will","be","writed",".","The","default","is","None","which","means","do","n't","write","to","file",".",":","param","no_cache",":","if","set","True",",","calculate","average","latency","by","iterating","all","the","samples","from","the","provided","dataloader","until","reaching","the","latency_sample_num",".","Default","set","to","be","False",",","meaning","always","loading","one","single","sample","from","cache","to","test","latency","."]
code: def optimize(
      self,
      model: nn.Module,
      training_data: Union[
          DataLoader, torch.Tensor, Tuple[torch.Tensor]
      ],
      validation_data: Optional[
          Union[DataLoader, torch.Tensor, Tuple[torch.Tensor]]
      ] = None,
      input_sample: Union[
          torch.Tensor, Dict, Tuple[torch.Tensor], None
      ] = None,
      metric: Optional[Callable] = None,
      direction: str = "max",
      thread_num: Optional[int] = None,
      accelerator: Optional[Tuple[str]] = None,
      precision: Optional[Tuple[str]] = None,
      use_ipex: Optional[bool] = None,
      jit_strict: Optional[bool] = True,
      enable_onednn: Optional[bool] = False,
      search_mode: str = "default",
      dynamic_axes: Union[bool, dict] = True,
      logging: bool = False,
      output_tensors: bool = True,
      latency_sample_num: int = 100,
      includes: Optional[List[str]] = None,
      excludes: Optional[List[str]] = None,
      output_filename: Optional[str] = None,
      no_cache: bool = False,
  ) -> None:
      """
      This function will give all available inference acceleration methods a try
      and record the latency, accuracy and model instance inside the Optimizer for
      future usage. All model instance is setting to eval mode.
  
      The available methods are "original", "fp32_channels_last", "fp32_ipex",
      "fp32_ipex_channels_last", "bf16", "bf16_channels_last", "bf16_ipex",
      "bf16_ipex_channels_last", "static_int8", "static_int8_ipex", "jit_fp32",
      "jit_fp32_channels_last", "jit_bf16", "jit_bf16_channels_last",
      "jit_fp32_ipex", "jit_fp32_ipex_channels_last", "jit_bf16_ipex",
      "jit_bf16_ipex_channels_last", "jit_int8", "jit_int8_channels_last",
      "openvino_fp32", "openvino_int8", "onnxruntime_fp32",
      "onnxruntime_int8_qlinear" and "onnxruntime_int8_integer".
  
      :param model: A torch.nn.Module to be optimized
      :param training_data: training_data support following formats:
  
              | 1. a torch.utils.data.dataloader.DataLoader object for training dataset.
              | Users should be careful with this parameter since this dataloader
              | might be exposed to the model, which causing data leak. The
              | batch_size of this dataloader is important as well, users may
              | want to set it to the same batch size you may want to use the model
              | in real deploy environment. E.g. batch size should be set to 1
              | if you would like to use the accelerated model in an online service.
              |
              | Each element in the DataLoader can be one of the following:
              |    a. a single Tensor or dict of Tensors
              |    b. a tuple:
              |         b1: if the length is 1, the first element will be treated as input
              |             to the model
              |         b2: if the length is 2, the first element will be treated as input
              |             to the model, with the sencond element treated as label.
              |             if the input to the model is a tuple, it will be unpacked as
              |             multiple inputs.
              |         b3: if the length is larger than 2, the first n elements as input
              |             to the model, with n being the argument lenth to the model.forward
              |             and the rest will be treated as label
              |
              | 2. a single element of the Dataloader specified above
  
      :param validation_data: (optional) validation_data is only needed when users care
                              about the possible accuracy drop. It support following formats:
  
              | 1. a torch.utils.data.dataloader.DataLoader object for accuracy evaluation.
              |
              | Each element in the DataLoader should be a tuple as least size of two:
              |     a: if the length is 2, the first element will be treated as input
              |        to the model, with the sencond element treated as label
              |     b: if the length is larger than 2, the first n elements as input
              |        to the model, with n being the argument lenth to the model.forward
              |        and the rest will be treated as label
              |
              | 2. a single element of the Dataloader specified above
  
      :param input_sample: (optional) A set of inputs for trace, defaults to None.
             In most cases, you don't need specify this parameter, it will be obtained from
             training_data. You have to specify this parameter only if the forward function
             of your model contains some kwargs like `def forward(self, x1, x2, x3=1)`.
      :param metric: (optional) A callable object which is used for calculating accuracy.
             It supports two kinds of callable object:
  
             | 1. A torchmetrics.Metric object or similar callable object which takes
             | prediction and target then returns an accuracy value in this calling
             | method `metric(pred, target)`. This requires data in validation_data
             | is composed of (input_data, target).
             |
             | 2. A callable object that takes model and validation_data (if
             | validation_data is not None) as input, and returns an accuracy value in
             | this calling method metric(model, data_loader) (or metric(model) if
             | validation_data is None). Note that there is no need to call `with
             | InferenceOptimizer.get_context()` in this object.
  
      :param direction: (optional) A string that indicates the higher/lower
             better for the metric, "min" for the lower the better and "max" for the
             higher the better. Default value is "max".
      :param thread_num: (optional) An int represents how many threads(cores) is needed for
             inference. This parameter only controls the usage of thread number in the process
             of latency calculation as well as later inference process of your obtained
             accelerated model. In other words, the process of model conversion and optional
             accuracy calculation won't be restricted by this parameter. Defaults to None,
             represents that all cores will be used.
      :param accelerator: (optional) A string tuple that specifies the accelerators to search.
             The optional accelerators are: None, 'openvino', 'onnxruntime', 'jit'.
             Defaults to None which represents there is no restriction on accelerators.
             If not None, then will only traverse corresponding methods whose accelerator falls
             within the specified accelerator tuple.
      :param precision: (optional) A string tuple that specifies the precision to search.
             The optional precision are: 'int8', 'bf16', and 'fp32'. Defaults to None which
             represents no precision limit. If not None, then will only traverse corresponding
             methods whose precision falls within the specified precision tuple.
      :param use_ipex: (optional) if not None, then will only try methods with/without
             this specific ipex setting.
      :param jit_strict: Whether recording your mutable container types. This parameter will be
             passed to ``torch.jit.trace``. if ``accelerator != 'jit'`` or
             ``jit_method='script'``, it will be ignored. Default to True.
      :param enable_onednn: Whether to use PyTorch JIT graph fuser based on oneDNN Graph API,
              which provides a flexible API for aggressive fusion. Default to
              ``False``, only valid when accelerator='jit', otherwise will
              be ignored. For more details, please refer https://github.com/
              pytorch/pytorch/tree/master/torch/csrc/jit/codegen/
              onednn#pytorch---onednn-graph-api-bridge.
      :param search_mode: Here are three modes for optimization:
  
             | 1. default: This mode only traverses a subset of all combinations. This subset
             | is a collection of methods that we select based on experience and think have
             | better acceleration effect in general. This mode allows you to quickly obtain a
             | good acceleration method, but it is not necessarily the global optimal. Default
             | to this mode if you don't specify accelerator/precision/use_ipex.
             |
             | 2. all: This mode will traverse all possible combinations, which can ensure
             | find the global optimization, but it will take a long time.
             |
             | 3. grid: If you have specified accelerator/precision/use_ipex, the default is
             | grid mode. We will sort and combine according to the value you specified to
             | get the search range.
  
      :param dynamic_axes: dict or boolean, default to True. By default the exported onnx model
             will have the first dim of each Tensor input as a dynamic batch_size. If
             dynamic_axes=False, the exported model will have the shapes of all input and output
             tensors set to exactly match those given in input_sample. To specify axes of
             tensors as dynamic (i.e. known only at run-time), set dynamic_axes to a dict with
             schema:
  
             | KEY (str): an input or output name. Each name must also be provided
             | in input_names or output_names.
             |
             | VALUE (dict or list): If a dict, keys are axis indices and values are
             | axis names. If a list, each element is an axis index.
  
             If accelerator != 'openvino'/'onnxruntime', it will be ignored.
      :param logging: whether to log detailed information of model conversion.
             Default: False.
      :param output_tensors: boolean, default to True and output of the model will be Tensors,
                             only valid when accelerator='onnxruntime' or accelerator='openvino',
                             otherwise will be ignored. If output_tensors=False, output of the
                             export model will be ndarray.
      :param latency_sample_num: (optional) a int represents the number of repetitions
             to calculate the average latency. The default value is 100.
      :param includes: (optional) a list of acceleration methods that will be included in the
             search. Default to None meaning including all available methods. "original" method
             will be automatically add to includes.
      :param excludes: (optional) a list of acceleration methods that will be excluded from the
             search. "original" will be ignored in the excludes.
      :param output_filename: (optional) a string filename is used to specify the file which the
             optimized table will be writed. The default is None which means don't write to file.
      :param no_cache: if set True, calculate average latency by iterating all the samples from
             the provided dataloader until reaching the latency_sample_num. Default set to be
             False, meaning always loading one single sample from cache to test latency.
      """
  
      # check if model is a nn.Module or inherited from a nn.Module
      invalidInputError(
          isinstance(model, nn.Module), "model should be a nn module."
      )
      invalidInputError(
          direction in ["min", "max"],
          "Only support direction 'min', 'max'.",
      )
      invalidInputError(
          accelerator is None or isinstance(accelerator, tuple),
          "accelerator must be a tuple.",
      )
      invalidInputError(
          precision is None or isinstance(precision, tuple),
          "precison must be a tuple.",
      )
      _check_accelerator = accelerator is None or all(
          ac in [None, "onnxruntime", "openvino", "jit"]
          for ac in accelerator
      )
      invalidInputError(
          _check_accelerator is True,
          "Only support accelerator None, 'onnxruntime', 'openvino' and 'jit'.",
      )
      _check_precision = precision is None or all(
          p in [None, "int8", "bf16", "fp32"] for p in precision
      )
      invalidInputError(
          _check_precision is True,
          "Only support precision 'int8', 'bf16', 'fp32'.",
      )
  
      if (
          accelerator is not None
          or precision is not None
          or use_ipex is not None
      ):
          search_mode = "grid"
          # setting search scope
          all_acceleration_methods = _obtain_combinations(
              self.ALL_INFERENCE_ACCELERATION_METHOD,
              precision,
              accelerator,
              use_ipex,
          )
      else:
          if search_mode == "all":
              all_acceleration_methods = (
                  self.ALL_INFERENCE_ACCELERATION_METHOD
              )
          elif search_mode == "default":
              # which is seting based on experience, and may need periodic update
              all_acceleration_methods = (
                  self.DEFAULT_INFERENCE_ACCELERATION_METHOD
              )
  
      # get the available methods whose dep is met
      available_dict: Dict = available_acceleration_combination(
          excludes=excludes,
          includes=includes,
          full_methods=all_acceleration_methods,
          all_methods=self.ALL_INFERENCE_ACCELERATION_METHOD,
      )
  
      self._direction: str = direction  # save direction as attr
      # record whether calculate accuracy in optimize by this attr
      if validation_data is None and metric is None:
          self._calculate_accuracy = False
      else:
          # test whether accuracy calculation works later
          self._calculate_accuracy = True
  
      default_threads: int = torch.get_num_threads()
      thread_num: int = None if thread_num is None else int(thread_num)
  
      result_map: Dict[str, Dict] = {}
  
      model.eval()  # change model to eval mode
  
      forward_args = get_forward_args(model)
      if input_sample is None:
          if isinstance(training_data, DataLoader):
              input_sample = get_input_example(
                  model, training_data, forward_args
              )
          else:
              if isinstance(training_data, Sequence):
                  if len(training_data) <= 2:
                      input_sample = training_data[0]
                      if len(training_data) == 2:
                          input_label = training_data[1]
                      else:
                          input_label = torch.Tensor([])
                  else:
                      input_sample = tuple(
                          training_data[: len(forward_args)]
                      )
                      input_label = tuple(
                          training_data[len(forward_args) :]
                      )
              else:
                  input_sample = training_data
                  input_label = torch.Tensor([])
              # turn training_data into dataset
              dataset = RepeatDataset(
                  sample=(input_sample, input_label), num=1
              )
              training_data = DataLoader(dataset, batch_size=1)
              training_data = remove_batch_dim_fn(training_data)
  
              if validation_data is not None and not isinstance(
                  validation_data, DataLoader
              ):
                  # turn validation_data into dataset
                  if isinstance(validation_data, Sequence):
                      if len(validation_data) <= 2:
                          val_sample = validation_data[0]
                          if len(validation_data) == 2:
                              val_label = validation_data[1]
                          else:
                              val_label = []
                      else:
                          val_sample = tuple(
                              validation_data[: len(forward_args)]
                          )
                          val_label = tuple(
                              validation_data[len(forward_args) :]
                          )
                  else:
                      val_sample = training_data
                      val_label = []
                  val_dataset = RepeatDataset(
                      sample=(val_sample, val_label), num=1
                  )
                  validation_data = DataLoader(
                      val_dataset, batch_size=1
                  )
                  validation_data = remove_batch_dim_fn(validation_data)
      # jit cannot handle `Mapping`, so we convert it to `dict`
      if isinstance(input_sample, Mapping):
          input_sample = dict(input_sample)
      st = time.perf_counter()
      try:
          with torch.no_grad():
              if isinstance(input_sample, (list, tuple)):
                  model(*input_sample)
              else:
                  if not isinstance(input_sample, (dict, torch.Tensor)):
                      warnings.warn(
                          "You may need to change `input_sample` to "
                          "a (list/tuple/dict of) Tensor to use jit."
                      )
                  model(input_sample)
      except Exception:
          invalidInputError(
              False,
              f"training_data is incompatible with your model input.",
          )
      baseline_time = time.perf_counter() - st
      if baseline_time > 0.1:  # 100ms
          sample_size_for_pot = 15
      else:
          sample_size_for_pot = 100
  
      # patch context manager
      model._nano_context_manager = generate_context_manager(
          accelerator=None, precision="fp32", thread_num=thread_num
      )
  
      print(
          "==========================Start Optimization=========================="
      )
      start_time = time.perf_counter()
      for idx, (method, available) in enumerate(available_dict.items()):
          result_map[method] = {}
          if available is False:
              result_map[method]["status"] = "lack dependency"
          else:
              print(
                  f"----------Start test {method} model "
                  f"({idx+1}/{len(available_dict)})----------"
              )
              option: AccelerationOption = (
                  self.ALL_INFERENCE_ACCELERATION_METHOD[method]
              )
              _precision = option.get_precision()
              try:
                  acce_model = option.optimize(
                      model,
                      training_data=training_data,
                      input_sample=input_sample,
                      thread_num=thread_num,
                      dynamic_axes=dynamic_axes,
                      logging=logging,
                      output_tensors=output_tensors,
                      sample_size_for_pot=sample_size_for_pot,
                      jit_strict=jit_strict,
                  )
              except Exception:
                  traceback.print_exc()
                  result_map[method]["status"] = "fail to convert"
                  print(
                      f"----------Failed to convert to {method}----------"
                  )
                  continue
  
              result_map[method]["status"] = "successful"
  
              def func_test(model, input_sample):
                  if isinstance(input_sample, (list, tuple)):
                      model(*input_sample)
                  else:
                      model(input_sample)
  
              with InferenceOptimizer.get_context(acce_model):
                  try:
                      if no_cache:
                          (
                              result_map[method]["latency"],
                              status,
                          ) = torch_loader_latency_calculate_helper(
                              latency_sample_num,
                              baseline_time,
                              func_test,
                              acce_model,
                              input_sample,
                              training_data,
                              forward_args,
                          )
                      else:
                          (
                              result_map[method]["latency"],
                              status,
                          ) = latency_calculate_helper(
                              latency_sample_num,
                              baseline_time,
                              func_test,
                              acce_model,
                              input_sample,
                          )
                      if status is False and method != "original":
                          result_map[method]["status"] = "early stopped"
                          # save model even early stop
                          result_map[method]["model"] = acce_model
                          torch.set_num_threads(default_threads)
                          continue
                  except Exception:
                      traceback.print_exc()
                      result_map[method]["status"] = "fail to forward"
                      print(
                          f"----------{method} failed to forward----------"
                      )
                      torch.set_num_threads(default_threads)
                      continue
  
                  torch.set_num_threads(default_threads)
                  if self._calculate_accuracy:
                      # here we suppose trace don't change accuracy,
                      # so we jump it to reduce time cost of optimize
                      if _precision == "fp32" and method != "original":
                          _accuracy = result_map["original"]["accuracy"]
                          if isinstance(_accuracy, torch.Tensor):
                              _accuracy = _accuracy.item()
                          _accuracy = sigfig.round(_accuracy, sigfigs=5)
                          result_map[method]["accuracy"] = (
                              str(_accuracy) + "*"
                          )
                      else:
                          if method == "original":
                              # test whether metric works
                              try:
                                  result_map[method][
                                      "accuracy"
                                  ] = _accuracy_calculate_helper(
                                      acce_model,
                                      metric,
                                      validation_data,
                                  )
                              except Exception:
                                  traceback.print_exc()
                                  self._calculate_accuracy = False
                                  invalidInputError(
                                      False,
                                      "Your metric is incompatible with validation_data or don't "
                                      "follow our given pattern. Our expected metric pattern is "
                                      "as follows:\n1. a torchmetrics.Metric object\n2. a "
                                      "callable object which takes prediction and target then "
                                      "returns a value in this calling method: `metric(pred, "
                                      "target)`\n3. a callable object that takes model and "
                                      "validation_data (if validation_data is not None) as input,"
                                      "and returns an accuracy value in this calling method: "
                                      "metric(model, data_loader) (or metric(model) if "
                                      "validation_data is None).",
                                  )
                          else:
                              result_map[method][
                                  "accuracy"
                              ] = _accuracy_calculate_helper(
                                  acce_model, metric, validation_data
                              )
                  else:
                      result_map[method]["accuracy"] = None
  
              result_map[method]["model"] = acce_model
              print(
                  f"----------Finish test {method} model "
                  f"({idx+1}/{len(available_dict)})----------"
              )
  
      self.optimized_model_dict: Dict = result_map
      print(
          "\n\n==========================Optimization Results=========================="
      )
  
      self._optimize_result = format_optimize_result(
          self.optimized_model_dict, self._calculate_accuracy
      )
      if self._calculate_accuracy:
          if precision is None or "fp32" in precision:
              # only show this line when there is traced model and metric value
              self._optimize_result += (
                  "* means we assume the metric value of the traced "
                  "model does not change, so we don't recompute metric value to save time.\n"
              )
      # save time cost to self._optimize_result
      time_cost = time.perf_counter() - start_time
      time_cost_str = f"Optimization cost {time_cost:.1f}s in total."
      self._optimize_result += time_cost_str
      if output_filename is not None:
          with open(output_filename, "w") as f:
              f.write(self._optimize_result)
      print(self._optimize_result)
      print(
          "===========================Stop Optimization==========================="
      )
  
imports: import torch
  from torch import nn
  import time
  import sigfig
  import multiprocessing
  from typing import Dict, Callable, Tuple, Optional, List, Union, Sequence, Mapping
  from torch.utils.data import DataLoader
  from torchmetrics.metric import Metric
  from bigdl.nano.utils.common import AccelerationOption, available_acceleration_combination, latency_calculate_helper, torch_loader_latency_calculate_helper, format_optimize_result, BaseInferenceOptimizer
  from bigdl.nano.utils.common import invalidInputError
  from bigdl.nano.pytorch.amp import BF16Model
  from bigdl.nano.pytorch.low_precision.jit_int8_api import PytorchJITINT8Model
  from bigdl.nano.deps.openvino.openvino_api import PytorchOpenVINOModel
  from bigdl.nano.deps.ipex.ipex_api import PytorchIPEXJITModel, PytorchIPEXJITBF16Model, PytorchIPEXQuantizationModel, PytorchIPEXPUModel
  from bigdl.nano.deps.onnxruntime.onnxruntime_api import PytorchONNXRuntimeModel
  from bigdl.nano.deps.neural_compressor.inc_api import quantize
  from bigdl.nano.pytorch.model import AcceleratedLightningModule
  from bigdl.nano.utils.pytorch import get_forward_args, get_input_example
  from bigdl.nano.utils.pytorch import NanoMetric
  from bigdl.nano.utils.pytorch import RepeatDataset, remove_batch_dim_fn
  from bigdl.nano.utils.pytorch import transform_multiple_input_dataloader_to_inc_mode, automatic_add_label_in_dataloader
  from bigdl.nano.utils.pytorch import TORCH_VERSION_LESS_1_10
  from bigdl.nano.utils.pytorch import save_model, load_model
  from bigdl.nano.utils.common import schedule_processors
  from bigdl.nano.utils.common import EnvContext
  from bigdl.nano.pytorch.context_manager import generate_context_manager, BaseContextManager, AutocastContextManager
  from .multi_instance import _MultiInstanceModel, _multi_instance_helper
  import traceback
  import warnings
  import os
label_desc: Use image node_id 1 for calling the InferenceOptimizer obj's underlying member method code with example usage: obj.optimize(model, training_data, validation_data, input_sample, metric, direction, thread_num, accelerator, precision, use_ipex, jit_strict, enable_onednn, search_mode, dynamic_axes, logging, output_tensors, latency_sample_num, includes, excludes, output_filename, no_cache) without return types