class_docstr: Forecaster base model for lstm, seq2seq, tcn and nbeats forecasters.
class_docstr_tok: ["Forecaster","base","model","for","lstm",",","seq2seq",",","tcn","and","nbeats","forecasters","."]
docstr: Evaluate using a trained forecaster.
  
  If you want to evaluate on a single node(which is common practice), please call
  .to_local().evaluate(data, ...)
  
  Please note that evaluate result is calculated by scaled y and yhat. If you scaled
  your data (e.g. use .scale() on the TSDataset), please follow the following code
  snap to evaluate your result if you need to evaluate on unscaled data.
  
  >>> from bigdl.chronos.metric.forecast_metrics import Evaluator
  >>> y_hat = forecaster.predict(x)
  >>> y_hat_unscaled = tsdata.unscale_numpy(y_hat) # or other customized unscale methods
  >>> y_unscaled = tsdata.unscale_numpy(y) # or other customized unscale methods
  >>> Evaluator.evaluate(metric=..., y_unscaled, y_hat_unscaled, multioutput=...)
  
  :param data: The data support following formats:
  
         | 1. a numpy ndarray tuple (x, y):
         | x's shape is (num_samples, lookback, feature_dim) where lookback and feature_dim
         | should be the same as past_seq_len and input_feature_num.
         | y's shape is (num_samples, horizon, target_dim), where horizon and target_dim
         | should be the same as future_seq_len and output_feature_num.
         |
         | 2. a xshard item:
         | each partition can be a dictionary of {'x': x, 'y': y}, where x and y's shape
         | should follow the shape stated before.
         |
         | 3. pytorch dataloader:
         | the dataloader should return x, y in each iteration with the shape as following:
         | x's shape is (num_samples, lookback, feature_dim) where lookback and feature_dim
         | should be the same as past_seq_len and input_feature_num.
         | y's shape is (num_samples, horizon, target_dim), where horizon and target_dim
         | should be the same as future_seq_len and output_feature_num.
         |
         | 4. A bigdl.chronos.data.tsdataset.TSDataset instance:
         | Forecaster will automatically process the TSDataset.
         | By default, TSDataset will be transformed to a pytorch dataloader,
         | which is memory-friendly while a little bit slower.
         | Users may call `roll` on the TSDataset before calling `fit`
         | Then the training speed will be faster but will consume more memory.
  
  :param batch_size: evaluate batch size. The value will not affect evaluate
         result but will affect resources cost(e.g. memory and time).
  :param multioutput: Defines aggregating of multiple output values.
         String in ['raw_values', 'uniform_average']. The value defaults to
         'raw_values'.The param is only effective when the forecaster is a
         non-distribtued version.
  :param quantize: if use the quantized model to predict.
  :param acceleration: bool variable indicates whether use original model.
         Default to True means use accelerated_model to predict, which requires
         to call one of .build_jit(), .build_onnx(), .build_openvino() and
         .optimize(), otherwise the original model will be used to predict.
  
  :return: A list of evaluation results. Each item represents a metric.
code: def evaluate(
      self,
      data,
      batch_size=32,
      multioutput="raw_values",
      quantize=False,
      acceleration: bool = True,
  ):
      """
      Evaluate using a trained forecaster.
  
      If you want to evaluate on a single node(which is common practice), please call
      .to_local().evaluate(data, ...)
  
      Please note that evaluate result is calculated by scaled y and yhat. If you scaled
      your data (e.g. use .scale() on the TSDataset), please follow the following code
      snap to evaluate your result if you need to evaluate on unscaled data.
  
      >>> from bigdl.chronos.metric.forecast_metrics import Evaluator
      >>> y_hat = forecaster.predict(x)
      >>> y_hat_unscaled = tsdata.unscale_numpy(y_hat) # or other customized unscale methods
      >>> y_unscaled = tsdata.unscale_numpy(y) # or other customized unscale methods
      >>> Evaluator.evaluate(metric=..., y_unscaled, y_hat_unscaled, multioutput=...)
  
      :param data: The data support following formats:
  
             | 1. a numpy ndarray tuple (x, y):
             | x's shape is (num_samples, lookback, feature_dim) where lookback and feature_dim
             | should be the same as past_seq_len and input_feature_num.
             | y's shape is (num_samples, horizon, target_dim), where horizon and target_dim
             | should be the same as future_seq_len and output_feature_num.
             |
             | 2. a xshard item:
             | each partition can be a dictionary of {'x': x, 'y': y}, where x and y's shape
             | should follow the shape stated before.
             |
             | 3. pytorch dataloader:
             | the dataloader should return x, y in each iteration with the shape as following:
             | x's shape is (num_samples, lookback, feature_dim) where lookback and feature_dim
             | should be the same as past_seq_len and input_feature_num.
             | y's shape is (num_samples, horizon, target_dim), where horizon and target_dim
             | should be the same as future_seq_len and output_feature_num.
             |
             | 4. A bigdl.chronos.data.tsdataset.TSDataset instance:
             | Forecaster will automatically process the TSDataset.
             | By default, TSDataset will be transformed to a pytorch dataloader,
             | which is memory-friendly while a little bit slower.
             | Users may call `roll` on the TSDataset before calling `fit`
             | Then the training speed will be faster but will consume more memory.
  
      :param batch_size: evaluate batch size. The value will not affect evaluate
             result but will affect resources cost(e.g. memory and time).
      :param multioutput: Defines aggregating of multiple output values.
             String in ['raw_values', 'uniform_average']. The value defaults to
             'raw_values'.The param is only effective when the forecaster is a
             non-distribtued version.
      :param quantize: if use the quantized model to predict.
      :param acceleration: bool variable indicates whether use original model.
             Default to True means use accelerated_model to predict, which requires
             to call one of .build_jit(), .build_onnx(), .build_openvino() and
             .optimize(), otherwise the original model will be used to predict.
  
      :return: A list of evaluation results. Each item represents a metric.
      """
      # data transform
      if isinstance(data, TSDataset):
          _rolled = data.numpy_x is None
          data = data.to_torch_data_loader(
              batch_size=batch_size,
              roll=_rolled,
              lookback=self.data_config["past_seq_len"],
              horizon=self.data_config["future_seq_len"],
              feature_col=data.roll_feature,
              target_col=data.roll_target,
              shuffle=False,
          )
      is_local_data = isinstance(data, (tuple, DataLoader))
      if not is_local_data and not self.distributed:
          data = xshard_to_np(data, mode="fit")
      if self.distributed:
          data = np_to_creator(data) if is_local_data else data
          return self.internal.evaluate(
              data=data, batch_size=batch_size
          )
      else:
          if not self.fitted:
              invalidInputError(
                  False,
                  "You must call fit or restore first before calling evaluate!",
              )
          if isinstance(data, DataLoader):
              from torch.utils.data.sampler import RandomSampler
  
              if isinstance(data.sampler, RandomSampler):
                  # If dataloader is shuffled, convert input_data to numpy()
                  # Avoid to iterate shuffled dataloader two times
                  input_x = []
                  input_y = []
                  for val in data:
                      input_x.append(val[0].numpy())
                      input_y.append(val[1].numpy())
                  input_data = np.concatenate(input_x, axis=0)
                  target = np.concatenate(input_y, axis=0)
              else:
                  input_data = data
                  target = np.concatenate(
                      tuple(val[1] for val in data), axis=0
                  )
          else:
              input_data, target = data
          if not self.context_enabled:
              self.cxt_manager = ForecasterContextManager(
                  self, self.thread_num, True
              )
          else:
              self.cxt_manager = DummyForecasterContextManager()
          with self.cxt_manager:
              if quantize:
                  if self.accelerate_method != "pytorch_int8":
                      invalidInputError(
                          False,
                          "Can't find the quantized model, "
                          "please call .quantize() method first",
                      )
                  yhat = _pytorch_fashion_inference(
                      model=self.accelerated_model,
                      input_data=input_data,
                      batch_size=batch_size,
                  )
              else:
                  if (
                      acceleration is False
                      or self.accelerated_model is None
                  ):
                      self.internal.eval()
                      yhat = _pytorch_fashion_inference(
                          model=self.internal,
                          input_data=input_data,
                          batch_size=batch_size,
                      )
                  else:
                      self.accelerated_model.eval()
                      yhat = _pytorch_fashion_inference(
                          model=self.accelerated_model,
                          input_data=input_data,
                          batch_size=batch_size,
                          output_tensor=self.optimized_model_output_tensor,
                      )
  
          aggregate = (
              "mean" if multioutput == "uniform_average" else None
          )
          return Evaluator.evaluate(
              self.metrics, target, yhat, aggregate=aggregate
          )
  
imports: from bigdl.chronos.forecaster.abstract import Forecaster
  from bigdl.chronos.forecaster.utils import *
  from bigdl.chronos.metric.forecast_metrics import Evaluator
  from typing import Optional
  import numpy
  import warnings
  import os
  from tempfile import TemporaryDirectory
  import torch
  from torch.utils.data import TensorDataset, DataLoader
  from .utils_hpo import GenericLightningModule, _format_metric_str, _config_has_search_space
  from bigdl.nano.utils.common import invalidOperationError, invalidInputError
  from bigdl.chronos.data.tsdataset import TSDataset
  from bigdl.chronos.pytorch.context_manager import DummyForecasterContextManager, ForecasterContextManager
  from bigdl.chronos.pytorch.utils import _pytorch_fashion_inference
label_desc: Use image node_id 12 for calling the BasePytorchForecaster obj's underlying member method code with example usage: obj.evaluate(data, batch_size, multioutput, quantize, acceleration) and returns: self, Evaluator