class_docstr: Class for benchmarking multiple ForecastConfig on a rolling window basis.
  
  Attributes
  ----------
  df : `pandas.DataFrame`
      Timeseries data to forecast.
      Contains columns [`time_col`, `value_col`], and optional regressor columns.
      Regressor columns should include future values for prediction.
  
  configs : `Dict` [`str`,  `ForecastConfig`]
      Dictionary of model configurations.
      A model configuration is a ``ForecastConfig``.
      See :class:`~greykite.framework.templates.autogen.forecast_config.ForecastConfig` for details on
      valid ``ForecastConfig``.
      Validity of the ``configs`` for benchmarking is checked via the ``validate`` method.
  
  tscv : `~greykite.sklearn.cross_validation.RollingTimeSeriesSplit`
      Cross-validation object that determines the rolling window evaluation.
      See :class:`~greykite.sklearn.cross_validation.RollingTimeSeriesSplit` for details.
      The ``forecast_horizon`` and ``periods_between_train_test`` parameters of ``configs`` are
      matched against that of ``tscv``. A ValueError is raised if there is a mismatch.
  
  forecaster : `~greykite.framework.templates.forecaster.Forecaster`
      Forecaster used to create the forecasts.
  
  is_run : bool, default False
      Indicator of whether the `run` method is executed.
      After executing `run`, this indicator is set to True.
      Some class methods like ``get_forecast`` requires ``is_run`` to be True
      to be executed.
  
  result : `dict`
      Stores the benchmarking results. Has the same keys as ``configs``.
  
  forecasts : `pandas.DataFrame`, default None
      Merged DataFrame of forecasts, upper and lower confidence interval for all
      input ``configs``. Also stores train end date and forecast step for each prediction.
docstr: Returns rolling average runtime in seconds for ``config_names``.
  
  Parameters
  ----------
  config_names : `list` [`str`], default None
      Which config results to plot. A list of config names.
      If None, uses all the available config keys.
  
  Returns
  -------
  runtimes_df : pd.DataFrame
      A DataFrame containing splitwise runtime in seconds for ``config_names``.
  
      For example. Let's assume::
  
          config_names = ["default_prophet", "custom_silverkite"]
          These are valid ``config_names`` and there are 2 splits for each.
  
          Then runtimes_df =
  
          config_name     split_num   runtime_sec
          default_prophet      0          *
          default_prophet      1          *
          custom_silverkite    0          *
          custom_silverkite    1          *
  
          where * represents computed values.
docstr_tok: ["Returns","rolling","average","runtime","in","seconds","for","``","config_names","``",".","Parameters","--","--","--","--","--","config_names",":","`","list","`","[","`","str","`","]",",","default","None","Which","config","results","to","plot",".","A","list","of","config","names",".","If","None",",","uses","all","the","available","config","keys",".","Returns","--","--","--","-","runtimes_df",":","pd.DataFrame","A","DataFrame","containing","splitwise","runtime","in","seconds","for","``","config_names","``",".","For","example",".","Let","'s","assume",":",":","config_names","=","[","``","default_prophet","''",",","``","custom_silverkite","''","]","These","are","valid","``","config_names","``","and","there","are","2","splits","for","each",".","Then","runtimes_df","=","config_name","split_num","runtime_sec","default_prophet","0","*","default_prophet","1","*","custom_silverkite","0","*","custom_silverkite","1","*","where","*","represents","computed","values","."]
code: def get_runtimes(self, config_names: List = None):
      """Returns rolling average runtime in seconds for ``config_names``.
  
      Parameters
      ----------
      config_names : `list` [`str`], default None
          Which config results to plot. A list of config names.
          If None, uses all the available config keys.
  
      Returns
      -------
      runtimes_df : pd.DataFrame
          A DataFrame containing splitwise runtime in seconds for ``config_names``.
  
          For example. Let's assume::
  
              config_names = ["default_prophet", "custom_silverkite"]
              These are valid ``config_names`` and there are 2 splits for each.
  
              Then runtimes_df =
  
              config_name     split_num   runtime_sec
              default_prophet      0          *
              default_prophet      1          *
              custom_silverkite    0          *
              custom_silverkite    1          *
  
              where * represents computed values.
      """
      if not self.is_run:
          raise ValueError(
              "Please execute the 'run' method to obtain runtimes."
          )
  
      config_names = self.get_valid_config_names(
          config_names=config_names
      )
      runtimes_df = pd.DataFrame()
      for config_name in config_names:
          rolling_evaluation = self.result[config_name][
              "rolling_evaluation"
          ]
          for num, (split_key, split_value) in enumerate(
              rolling_evaluation.items()
          ):
              split_runtime_df = pd.DataFrame(
                  {
                      "config_name": config_name,
                      "split_num": num,
                      "runtime_sec": split_value["runtime_sec"],
                  },
                  index=[num],
              )
              runtimes_df = pd.concat([runtimes_df, split_runtime_df])
  
      return runtimes_df.reset_index(drop=True)
  
imports: from typing import Dict
  from typing import List
  import numpy
  import pandas
  import plotly.graph_objects
  from plotly.colors import DEFAULT_PLOTLY_COLORS
  from tqdm.autonotebook import tqdm
  from greykite.common.constants import ACTUAL_COL
  from greykite.common.constants import PREDICTED_COL
  from greykite.common.constants import PREDICTED_LOWER_COL
  from greykite.common.constants import PREDICTED_UPPER_COL
  from greykite.common.constants import TIME_COL
  from greykite.common.constants import VALUE_COL
  from greykite.common.evaluation import EvaluationMetricEnum
  from greykite.common.evaluation import add_finite_filter_to_scorer
  from greykite.common.logging import LoggingLevelEnum
  from greykite.common.logging import log_message
  from greykite.common.python_utils import get_pattern_cols
  from greykite.common.viz.timeseries_plotting import plot_multivariate
  from greykite.common.viz.timeseries_plotting import plot_multivariate_grouped
  from greykite.framework.benchmark.benchmark_class_helper import forecast_pipeline_rolling_evaluation
  from greykite.framework.constants import FORECAST_STEP_COL
  from greykite.framework.templates.autogen.forecast_config import ForecastConfig
  from greykite.framework.templates.forecaster import Forecaster
  from greykite.sklearn.cross_validation import RollingTimeSeriesSplit
label_desc: Use image node_id 11 for calling the BenchmarkForecastConfig obj's underlying member method code with example usage: obj.get_runtimes(config_names) and returns: runtimes_df