class_docstr: Class for benchmarking multiple ForecastConfig on a rolling window basis.
  
  Attributes
  ----------
  df : `pandas.DataFrame`
      Timeseries data to forecast.
      Contains columns [`time_col`, `value_col`], and optional regressor columns.
      Regressor columns should include future values for prediction.
  
  configs : `Dict` [`str`,  `ForecastConfig`]
      Dictionary of model configurations.
      A model configuration is a ``ForecastConfig``.
      See :class:`~greykite.framework.templates.autogen.forecast_config.ForecastConfig` for details on
      valid ``ForecastConfig``.
      Validity of the ``configs`` for benchmarking is checked via the ``validate`` method.
  
  tscv : `~greykite.sklearn.cross_validation.RollingTimeSeriesSplit`
      Cross-validation object that determines the rolling window evaluation.
      See :class:`~greykite.sklearn.cross_validation.RollingTimeSeriesSplit` for details.
      The ``forecast_horizon`` and ``periods_between_train_test`` parameters of ``configs`` are
      matched against that of ``tscv``. A ValueError is raised if there is a mismatch.
  
  forecaster : `~greykite.framework.templates.forecaster.Forecaster`
      Forecaster used to create the forecasts.
  
  is_run : bool, default False
      Indicator of whether the `run` method is executed.
      After executing `run`, this indicator is set to True.
      Some class methods like ``get_forecast`` requires ``is_run`` to be True
      to be executed.
  
  result : `dict`
      Stores the benchmarking results. Has the same keys as ``configs``.
  
  forecasts : `pandas.DataFrame`, default None
      Merged DataFrame of forecasts, upper and lower confidence interval for all
      input ``configs``. Also stores train end date and forecast step for each prediction.
docstr: Returns splitwise rolling evaluation metric values.
   These values are grouped by the grouping method chosen by ``groupby_time_feature``,
   ``groupby_sliding_window_size`` and ``groupby_custom_column``.
  See `~greykite.framework.output.univariate_forecast.UnivariateForecast.get_grouping_evaluation`
  for details on grouping method.
  
   Parameters
  ----------
  metric_dict : `dict` [`str`, `callable`]
      Evaluation metrics to compute. Same as
      `~greykite.framework.framework.benchmark.benchmark_class.BenchmarkForecastConfig.get_evaluation_metrics`.
  config_names : `list` [`str`], default None
      Which config results to plot. A list of config names.
      If None, uses all the available config keys.
  which: `str`
      "train" or "test". Which dataset to evaluate.
  groupby_time_feature : `str` or None, default None
      If provided, groups by a column generated by
      `~greykite.common.features.timeseries_features.build_time_features_df`.
      See that function for valid values.
  groupby_sliding_window_size : `int` or None, default None
      If provided, sequentially partitions data into groups of size
      ``groupby_sliding_window_size``.
  groupby_custom_column : `pandas.Series` or None, default None
      If provided, groups by this column value. Should be same length as the DataFrame.
  
  Returns
  -------
  grouped_evaluation_df : `pandas.DataFrame`
      A DataFrame containing splitwise train and test evaluation metrics for ``metric_dict``
      and ``config_names``. The evaluation metrics are grouped by the grouping method.
docstr_tok: ["Returns","splitwise","rolling","evaluation","metric","values",".","These","values","are","grouped","by","the","grouping","method","chosen","by","``","groupby_time_feature","``",",","``","groupby_sliding_window_size","``","and","``","groupby_custom_column","``",".","See","`","~greykite.framework.output.univariate_forecast.UnivariateForecast.get_grouping_evaluation","`","for","details","on","grouping","method",".","Parameters","--","--","--","--","--","metric_dict",":","`","dict","`","[","`","str","`",",","`","callable","`","]","Evaluation","metrics","to","compute",".","Same","as","`","~greykite.framework.framework.benchmark.benchmark_class.BenchmarkForecastConfig.get_evaluation_metrics","`",".","config_names",":","`","list","`","[","`","str","`","]",",","default","None","Which","config","results","to","plot",".","A","list","of","config","names",".","If","None",",","uses","all","the","available","config","keys",".","which",":","`","str","`","``","train","''","or","``","test","''",".","Which","dataset","to","evaluate",".","groupby_time_feature",":","`","str","`","or","None",",","default","None","If","provided",",","groups","by","a","column","generated","by","`","~greykite.common.features.timeseries_features.build_time_features_df","`",".","See","that","function","for","valid","values",".","groupby_sliding_window_size",":","`","int","`","or","None",",","default","None","If","provided",",","sequentially","partitions","data","into","groups","of","size","``","groupby_sliding_window_size","``",".","groupby_custom_column",":","`","pandas.Series","`","or","None",",","default","None","If","provided",",","groups","by","this","column","value",".","Should","be","same","length","as","the","DataFrame",".","Returns","--","--","--","-","grouped_evaluation_df",":","`","pandas.DataFrame","`","A","DataFrame","containing","splitwise","train","and","test","evaluation","metrics","for","``","metric_dict","``","and","``","config_names","``",".","The","evaluation","metrics","are","grouped","by","the","grouping","method","."]
code: def get_grouping_evaluation_metrics(
      self,
      metric_dict: Dict,
      config_names: List = None,
      which: str = "train",
      groupby_time_feature: str = None,
      groupby_sliding_window_size: int = None,
      groupby_custom_column: pd.Series = None,
  ):
      """Returns splitwise rolling evaluation metric values.
       These values are grouped by the grouping method chosen by ``groupby_time_feature``,
       ``groupby_sliding_window_size`` and ``groupby_custom_column``.
      See `~greykite.framework.output.univariate_forecast.UnivariateForecast.get_grouping_evaluation`
      for details on grouping method.
  
       Parameters
      ----------
      metric_dict : `dict` [`str`, `callable`]
          Evaluation metrics to compute. Same as
          `~greykite.framework.framework.benchmark.benchmark_class.BenchmarkForecastConfig.get_evaluation_metrics`.
      config_names : `list` [`str`], default None
          Which config results to plot. A list of config names.
          If None, uses all the available config keys.
      which: `str`
          "train" or "test". Which dataset to evaluate.
      groupby_time_feature : `str` or None, default None
          If provided, groups by a column generated by
          `~greykite.common.features.timeseries_features.build_time_features_df`.
          See that function for valid values.
      groupby_sliding_window_size : `int` or None, default None
          If provided, sequentially partitions data into groups of size
          ``groupby_sliding_window_size``.
      groupby_custom_column : `pandas.Series` or None, default None
          If provided, groups by this column value. Should be same length as the DataFrame.
  
      Returns
      -------
      grouped_evaluation_df : `pandas.DataFrame`
          A DataFrame containing splitwise train and test evaluation metrics for ``metric_dict``
          and ``config_names``. The evaluation metrics are grouped by the grouping method.
      """
      if not self.is_run:
          raise ValueError(
              "Please execute the 'run' method before computing "
              "grouped evaluation metrics."
          )
  
      metric_dict = self.autocomplete_metric_dict(
          metric_dict=metric_dict, enum_class=EvaluationMetricEnum
      )
  
      config_names = self.get_valid_config_names(
          config_names=config_names
      )
  
      grouped_evaluation_df = pd.DataFrame()
      for config_name in config_names:
          rolling_evaluation = self.result[config_name][
              "rolling_evaluation"
          ]
          for num, (split_key, split_value) in enumerate(
              rolling_evaluation.items()
          ):
              forecast = split_value["pipeline_result"].forecast
              split_evaluation_df = pd.DataFrame()
              for metric_name, metric_func in metric_dict.items():
                  grouped_df = forecast.get_grouping_evaluation(
                      score_func=metric_func,
                      score_func_name=metric_name,
                      which=which,
                      groupby_time_feature=groupby_time_feature,
                      groupby_sliding_window_size=groupby_sliding_window_size,
                      groupby_custom_column=groupby_custom_column,
                  )
                  # Adds grouped_df to split_evaluation_df, handling the case if split_evaluation_df is empty
                  # If the actual values are missing, grouped_df.shape[0] might be 0
                  if grouped_df.shape[0] > 0:
                      if split_evaluation_df.empty:
                          split_evaluation_df = grouped_df
                      else:
                          groupby_col = split_evaluation_df.columns[0]
                          split_evaluation_df = pd.merge(
                              split_evaluation_df,
                              grouped_df,
                              on=groupby_col,
                          )
                  else:
                      # This column name is the same as that obtained from
                      # `~greykite.framework.output.univariate_forecast.UnivariateForecast.get_grouping_evaluation`
                      split_evaluation_df[
                          f"{which} {metric_name}"
                      ] = np.nan
              split_evaluation_df.insert(0, "config_name", config_name)
              split_evaluation_df.insert(1, "split_num", num)
              grouped_evaluation_df = pd.concat(
                  [grouped_evaluation_df, split_evaluation_df]
              )
      grouped_evaluation_df = grouped_evaluation_df.reset_index(
          drop=True
      )
  
      return grouped_evaluation_df
  
imports: from typing import Dict
  from typing import List
  import numpy
  import pandas
  import plotly.graph_objects
  from plotly.colors import DEFAULT_PLOTLY_COLORS
  from tqdm.autonotebook import tqdm
  from greykite.common.constants import ACTUAL_COL
  from greykite.common.constants import PREDICTED_COL
  from greykite.common.constants import PREDICTED_LOWER_COL
  from greykite.common.constants import PREDICTED_UPPER_COL
  from greykite.common.constants import TIME_COL
  from greykite.common.constants import VALUE_COL
  from greykite.common.evaluation import EvaluationMetricEnum
  from greykite.common.evaluation import add_finite_filter_to_scorer
  from greykite.common.logging import LoggingLevelEnum
  from greykite.common.logging import log_message
  from greykite.common.python_utils import get_pattern_cols
  from greykite.common.viz.timeseries_plotting import plot_multivariate
  from greykite.common.viz.timeseries_plotting import plot_multivariate_grouped
  from greykite.framework.benchmark.benchmark_class_helper import forecast_pipeline_rolling_evaluation
  from greykite.framework.constants import FORECAST_STEP_COL
  from greykite.framework.templates.autogen.forecast_config import ForecastConfig
  from greykite.framework.templates.forecaster import Forecaster
  from greykite.sklearn.cross_validation import RollingTimeSeriesSplit
label_desc: Use image node_id 9 for calling the BenchmarkForecastConfig obj's underlying member method code with example usage: obj.get_grouping_evaluation_metrics(metric_dict, config_names, which, groupby_time_feature, groupby_sliding_window_size, groupby_custom_column) and returns: grouped_evaluation_df