docstr: This function will give all available inference acceleration methods a try
  and record the latency, accuracy and model instance inside the Optimizer for
  future usage. All model instance is setting to eval mode.
  
  The available methods are "original", "openvino_fp32", "onnxruntime_fp32", "int8".
  
  :param model: A keras.Model to be optimized
  :param x: Input data which is used for training. It could be:
  
            | 1. a Numpy array (or array-like), or a list of arrays (in case the model
            | has multiple inputs).
            |
            | 2. a TensorFlow tensor, or a list of tensors (in case the model has
            | multiple inputs).
            |
            | 3. an unbatched tf.data.Dataset. Should return a tuple of (inputs, targets).
  
            X will be used as calibration dataset for Post-Training Static Quantization (PTQ),
            as well as be used for generating input_sample to calculate latency.
            To avoid data leak during calibration, please use training dataset.
  :param y: Target data. Like the input data x, it could be either Numpy array(s) or
            TensorFlow tensor(s). Its length should be consistent with x.
            If x is a dataset, y will be ignored (since targets will be obtained from x).
  :param validation_data: (optional) An unbatched tf.data.Dataset object for accuracy
         evaluation. This is only needed when users care about the possible accuracy drop.
  :param input_spec: (optional) A (tuple or list of) ``tf.TensorSpec``
                     defining the shape/dtype of the input. This is only required when
                     you have a custom Keras model (no input/output layer is explicitly
                     defined).
  :param metric: (optional) A tensorflow.keras.metrics.Metric object which is used for
         calculating accuracy.
  :param direction: (optional) A string that indicates the higher/lower
         better for the metric, "min" for the lower the better and "max" for the
         higher the better. Default value is "max".
  :param thread_num: (optional) An int represents how many threads(cores) is needed for
         inference. This parameter only controls the usage of thread number in the process
         of latency calculation as well as later inference process of your obtained
         accelerated model. In other words, the process of model conversion and optional
         accuracy calculation won't be restricted by this parameter. Defaults to None,
         represents that all cores will be used.
  :param logging: whether to log detailed information of model conversion.
         Default: False.
  :param latency_sample_num: (optional) a int represents the number of repetitions
         to calculate the average latency. The default value is 100.
  :param includes: (optional) a list of acceleration methods that will be included in the
         search. Default to None meaning including all available methods. "original" method
         will be automatically add to includes.
  :param excludes: (optional) a list of acceleration methods that will be excluded from the
         search. "original" will be ignored in the excludes.
  :param output_filename: (optional) a string filename is used to specify the file which the
         optimized table will be writed. The default is None which means don't write to file.
docstr_tok: ["This","function","will","give","all","available","inference","acceleration","methods","a","try","and","record","the","latency",",","accuracy","and","model","instance","inside","the","Optimizer","for","future","usage",".","All","model","instance","is","setting","to","eval","mode",".","The","available","methods","are","``","original","''",",","``","openvino_fp32","''",",","``","onnxruntime_fp32","''",",","``","int8","''",".",":","param","model",":","A","keras.Model","to","be","optimized",":","param","x",":","Input","data","which","is","used","for","training",".","It","could","be",":","|","1.","a","Numpy","array","(","or","array-like",")",",","or","a","list","of","arrays","(","in","case","the","model","|","has","multiple","inputs",")",".","|","|","2.","a","TensorFlow","tensor",",","or","a","list","of","tensors","(","in","case","the","model","has","|","multiple","inputs",")",".","|","|","3.","an","unbatched","tf.data.Dataset",".","Should","return","a","tuple","of","(","inputs",",","targets",")",".","X","will","be","used","as","calibration","dataset","for","Post-Training","Static","Quantization","(","PTQ",")",",","as","well","as","be","used","for","generating","input_sample","to","calculate","latency",".","To","avoid","data","leak","during","calibration",",","please","use","training","dataset",".",":","param","y",":","Target","data",".","Like","the","input","data","x",",","it","could","be","either","Numpy","array","(","s",")","or","TensorFlow","tensor","(","s",")",".","Its","length","should","be","consistent","with","x",".","If","x","is","a","dataset",",","y","will","be","ignored","(","since","targets","will","be","obtained","from","x",")",".",":","param","validation_data",":","(","optional",")","An","unbatched","tf.data.Dataset","object","for","accuracy","evaluation",".","This","is","only","needed","when","users","care","about","the","possible","accuracy","drop",".",":","param","input_spec",":","(","optional",")","A","(","tuple","or","list","of",")","``","tf.TensorSpec","``","defining","the","shape\/dtype","of","the","input",".","This","is","only","required","when","you","have","a","custom","Keras","model","(","no","input\/output","layer","is","explicitly","defined",")",".",":","param","metric",":","(","optional",")","A","tensorflow.keras.metrics.Metric","object","which","is","used","for","calculating","accuracy",".",":","param","direction",":","(","optional",")","A","string","that","indicates","the","higher\/lower","better","for","the","metric",",","``","min","''","for","the","lower","the","better","and","``","max","''","for","the","higher","the","better",".","Default","value","is","``","max","''",".",":","param","thread_num",":","(","optional",")","An","int","represents","how","many","threads","(","cores",")","is","needed","for","inference",".","This","parameter","only","controls","the","usage","of","thread","number","in","the","process","of","latency","calculation","as","well","as","later","inference","process","of","your","obtained","accelerated","model",".","In","other","words",",","the","process","of","model","conversion","and","optional","accuracy","calculation","wo","n't","be","restricted","by","this","parameter",".","Defaults","to","None",",","represents","that","all","cores","will","be","used",".",":","param","logging",":","whether","to","log","detailed","information","of","model","conversion",".","Default",":","False",".",":","param","latency_sample_num",":","(","optional",")","a","int","represents","the","number","of","repetitions","to","calculate","the","average","latency",".","The","default","value","is","100.",":","param","includes",":","(","optional",")","a","list","of","acceleration","methods","that","will","be","included","in","the","search",".","Default","to","None","meaning","including","all","available","methods",".","``","original","''","method","will","be","automatically","add","to","includes",".",":","param","excludes",":","(","optional",")","a","list","of","acceleration","methods","that","will","be","excluded","from","the","search",".","``","original","''","will","be","ignored","in","the","excludes",".",":","param","output_filename",":","(","optional",")","a","string","filename","is","used","to","specify","the","file","which","the","optimized","table","will","be","writed",".","The","default","is","None","which","means","do","n't","write","to","file","."]
code: def optimize(
      self,
      model: Model,
      x: Union[tf.Tensor, np.ndarray, tf.data.Dataset],
      y: Union[tf.Tensor, np.ndarray] = None,
      validation_data: Optional[Dataset] = None,
      input_spec=None,
      batch_size: int = 1,
      metric: Optional[Metric] = None,
      direction: str = "max",
      thread_num: Optional[int] = None,
      logging: bool = False,
      latency_sample_num: int = 100,
      includes: Optional[List[str]] = None,
      excludes: Optional[List[str]] = None,
      output_filename: Optional[str] = None,
  ) -> None:
      """
      This function will give all available inference acceleration methods a try
      and record the latency, accuracy and model instance inside the Optimizer for
      future usage. All model instance is setting to eval mode.
  
      The available methods are "original", "openvino_fp32", "onnxruntime_fp32", "int8".
  
      :param model: A keras.Model to be optimized
      :param x: Input data which is used for training. It could be:
  
                | 1. a Numpy array (or array-like), or a list of arrays (in case the model
                | has multiple inputs).
                |
                | 2. a TensorFlow tensor, or a list of tensors (in case the model has
                | multiple inputs).
                |
                | 3. an unbatched tf.data.Dataset. Should return a tuple of (inputs, targets).
  
                X will be used as calibration dataset for Post-Training Static Quantization (PTQ),
                as well as be used for generating input_sample to calculate latency.
                To avoid data leak during calibration, please use training dataset.
      :param y: Target data. Like the input data x, it could be either Numpy array(s) or
                TensorFlow tensor(s). Its length should be consistent with x.
                If x is a dataset, y will be ignored (since targets will be obtained from x).
      :param validation_data: (optional) An unbatched tf.data.Dataset object for accuracy
             evaluation. This is only needed when users care about the possible accuracy drop.
      :param input_spec: (optional) A (tuple or list of) ``tf.TensorSpec``
                         defining the shape/dtype of the input. This is only required when
                         you have a custom Keras model (no input/output layer is explicitly
                         defined).
      :param metric: (optional) A tensorflow.keras.metrics.Metric object which is used for
             calculating accuracy.
      :param direction: (optional) A string that indicates the higher/lower
             better for the metric, "min" for the lower the better and "max" for the
             higher the better. Default value is "max".
      :param thread_num: (optional) An int represents how many threads(cores) is needed for
             inference. This parameter only controls the usage of thread number in the process
             of latency calculation as well as later inference process of your obtained
             accelerated model. In other words, the process of model conversion and optional
             accuracy calculation won't be restricted by this parameter. Defaults to None,
             represents that all cores will be used.
      :param logging: whether to log detailed information of model conversion.
             Default: False.
      :param latency_sample_num: (optional) a int represents the number of repetitions
             to calculate the average latency. The default value is 100.
      :param includes: (optional) a list of acceleration methods that will be included in the
             search. Default to None meaning including all available methods. "original" method
             will be automatically add to includes.
      :param excludes: (optional) a list of acceleration methods that will be excluded from the
             search. "original" will be ignored in the excludes.
      :param output_filename: (optional) a string filename is used to specify the file which the
             optimized table will be writed. The default is None which means don't write to file.
      """
      # check if model is a nn.Module or inherited from a nn.Module
      invalidInputError(
          isinstance(model, Model), "model should be a Keras Model."
      )
      invalidInputError(
          direction in ["min", "max"],
          "Only support direction 'min', 'max'.",
      )
  
      # get the available methods whose dep is met
      available_dict: Dict = available_acceleration_combination(
          excludes=excludes,
          includes=includes,
          full_methods=self.ALL_INFERENCE_ACCELERATION_METHOD,
      )
  
      self._direction: str = direction  # save direction as attr
      # record whether calculate accuracy in optimize by this attr
      if validation_data is None or metric is None:
          self._calculate_accuracy = False
      else:
          # test whether accuracy calculation works later
          # make sure dataset don't have batch
          batched_validation_data = validation_data.batch(batch_size)
          self._calculate_accuracy = True
  
      if os.getenv("OMP_NUM_THREADS") is not None:
          default_threads: int = int(os.getenv("OMP_NUM_THREADS"))  # type: ignore
      else:
          default_threads = None  # type: ignore
      thread_num = default_threads if thread_num is None else int(thread_num)  # type: ignore
  
      result_map: Dict[str, Dict] = {}
  
      if isinstance(x, Dataset):
          batched_training_dataset = x.batch(batch_size)
          input_sample = next(iter(batched_training_dataset))
          # todo: for now, if len(batch_data) == 2 we assume it is (x, y),
          # otherwise, we assume it is x or (x1, x2, x3, ...)
          if (
              isinstance(input_sample, (list, tuple))
              and len(input_sample) == 2
          ):
              input_sample = input_sample[:-1]
      else:
          input_sample = tf.convert_to_tensor(x[:batch_size])
  
      if (
          isinstance(input_sample, (list, tuple))
          and len(input_sample) == 1
      ):
          input_sample = input_sample[0]
  
      st = time.perf_counter()
      try:
          if isinstance(input_sample, tf.Tensor):
              model(input_sample)
          else:
              model(*input_sample)
      except Exception:
          invalidInputError(
              False, "x is incompatible with your model input."
          )
      baseline_time = time.perf_counter() - st
      if baseline_time > 0.1:  # 100ms
          sample_size_for_pot = 15
      else:
          sample_size_for_pot = 100
  
      print(
          "==========================Start Optimization=========================="
      )
      start_time = time.perf_counter()
      for idx, (method, available) in enumerate(available_dict.items()):
          result_map[method] = {}
          if available is False:
              result_map[method]["status"] = "lack dependency"
          else:
              print(
                  f"----------Start test {method} model "
                  f"({idx+1}/{len(available_dict)})----------"
              )
              option: AccelerationOption = (
                  self.ALL_INFERENCE_ACCELERATION_METHOD[method]
              )
              precision: str = option.get_precision()
              try:
                  acce_model = option.optimize(
                      model=model,
                      x=x,
                      y=y,
                      input_spec=input_spec,
                      thread_num=thread_num,
                      logging=logging,
                      sample_size_for_pot=sample_size_for_pot,
                  )
              except Exception:
                  traceback.print_exc()
                  result_map[method]["status"] = "fail to convert"
                  print(
                      f"----------Failed to convert to {method}----------"
                  )
                  continue
  
              result_map[method]["status"] = "successful"
  
              def func_test(model, *args):
                  model(*args)
  
              try:
                  if (
                      method in ("original", "static_int8")
                      and thread_num is not None
                  ):
                      _flag = True  # represent whether subprocess works
                      # for original keras model, as tf.config.threading can't set thread
                      # during running, so here we use subprocess to calculate throughput
                      params = {
                          "iterrun": latency_sample_num,
                          "func": func_test,
                          "model": model,  # save original model
                          "input_sample": input_sample,
                          "method": method,
                      }
                      with tempfile.TemporaryDirectory() as temp_dir:
                          if method != "original":
                              # save accelerated model
                              InferenceOptimizer.save(
                                  acce_model, temp_dir
                              )
                          _filename = os.path.join(temp_dir, "params")
                          cloudpickle.dump(
                              params, open(_filename, "wb")
                          )
                          my_env = os.environ.copy()
                          my_env["OMP_NUM_THREADS"] = str(thread_num)
                          worker_file = os.path.join(
                              os.path.split(os.path.realpath(__file__))[
                                  0
                              ],
                              "_worker.py",
                          )
                          try:
                              result = subprocess.run(
                                  [
                                      "python",
                                      worker_file,
                                      _filename,
                                      str(thread_num),
                                  ],
                                  capture_output=True,
                                  universal_newlines=True,
                                  env=my_env,
                              )
                              latency = float(result.stdout.strip())
                              result_map[method]["latency"] = latency
                          except Exception:
                              _flag = False
                  if (
                      method != "original"
                      or thread_num is None
                      or _flag is False
                  ):
                      if isinstance(input_sample, tf.Tensor):
                          (
                              result_map[method]["latency"],
                              status,
                          ) = latency_calculate_helper(
                              latency_sample_num,
                              baseline_time,
                              func_test,
                              acce_model,
                              input_sample,
                          )
                      else:
                          (
                              result_map[method]["latency"],
                              status,
                          ) = latency_calculate_helper(
                              latency_sample_num,
                              baseline_time,
                              func_test,
                              acce_model,
                              *input_sample,
                          )
                      if status is False and method != "original":
                          result_map[method]["status"] = "early stopped"
                          continue
              except Exception:
                  traceback.print_exc()
                  result_map[method]["status"] = "fail to forward"
                  print(
                      f"----------{method} failed to forward----------"
                  )
                  continue
  
              if self._calculate_accuracy:
                  # here we suppose trace don't change accuracy,
                  # so we jump it to reduce time cost of optimize
                  if precision == "fp32" and method != "original":
                      _accuracy = result_map["original"]["accuracy"]
                      _accuracy = sigfig.round(_accuracy, sigfigs=5)
                      result_map[method]["accuracy"] = (
                          str(_accuracy) + "*"
                      )
                  else:
                      if method == "original":
                          # test whether metric works
                          try:
                              result_map[method][
                                  "accuracy"
                              ] = _accuracy_calculate_helper(
                                  acce_model,
                                  metric,
                                  batched_validation_data,
                              )
                          except Exception:
                              traceback.print_exc()
                              self._calculate_accuracy = False
                      else:
                          result_map[method][
                              "accuracy"
                          ] = _accuracy_calculate_helper(
                              acce_model,
                              metric,
                              batched_validation_data,
                          )
              else:
                  result_map[method]["accuracy"] = None
  
              result_map[method]["model"] = acce_model
              print(
                  f"----------Finish test {method} model "
                  f"({idx+1}/{len(available_dict)})----------"
              )
  
      self.optimized_model_dict: Dict = result_map
      print(
          "\n\n==========================Optimization Results=========================="
      )
  
      self._optimize_result = format_optimize_result(
          self.optimized_model_dict, self._calculate_accuracy
      )
      if self._calculate_accuracy:
          # only show this line when there is accuracy data
          self._optimize_result += (
              "* means we assume the metric value of the traced "
              "model does not change, so we don't recompute metric value to save time.\n"
          )
      # save time cost to self._optimize_result
      time_cost = time.perf_counter() - start_time
      time_cost_str = f"Optimization cost {time_cost:.1f}s in total."
      self._optimize_result += time_cost_str
      if output_filename is not None:
          with open(output_filename, "w") as f:
              f.write(self._optimize_result)
      print(self._optimize_result)
      print(
          "===========================Stop Optimization==========================="
      )
  
imports: import os
  import subprocess
  import tempfile
  import cloudpickle
  import copy
  import time
  import operator
  from pathlib import Path
  import numpy
  import traceback
  import inspect
  import sigfig
  import tensorflow
  import keras
  from typing import Dict, Optional, List, Union, Callable
  from bigdl.nano.utils.common import BaseInferenceOptimizer, available_acceleration_combination, AccelerationOption, latency_calculate_helper, format_optimize_result
  from bigdl.nano.utils.common import invalidInputError
  from bigdl.nano.utils.tf import _ModuleWrapper
  from bigdl.nano.utils.tf import patch_compiled_and_attrs, patch_attrs
  from tensorflow.keras import Model
  from tensorflow.data import Dataset
  from tensorflow.keras.metrics import Metric
  from bigdl.nano.deps.neural_compressor.inc_api import quantize
  from bigdl.nano.deps.openvino.openvino_api import KerasOpenVINOModel
  from bigdl.nano.deps.onnxruntime.onnxruntime_api import KerasONNXRuntimeModel
  from bigdl.nano.deps.openvino.openvino_api import load_openvino_model
  from bigdl.nano.deps.onnxruntime.onnxruntime_api import load_onnxruntime_model
  from bigdl.nano.deps.neural_compressor.inc_api import load_inc_model
  from bigdl.nano.tf.keras.amp import BF16Model, load_bf16_model
  from bigdl.nano.utils.common import compare_version
  from bigdl.nano.utils.tf import try_fake_inference
label_desc: Use image node_id 1 for calling the InferenceOptimizer obj's underlying member method code with example usage: obj.optimize(model, x, y, validation_data, input_spec, batch_size, metric, direction, thread_num, logging, latency_sample_num, includes, excludes, output_filename) without return types