class_docstr: Implements methods to automatically infer holiday effects.
  
  The class works for daily and sub-daily data.
  Sub-daily data is aggregated into daily data.
  It pulls holiday candidates from `pypi:holidays-ext`,
  and adds a pre-specified number of days before/after the holiday candidates
  as the whole holiday candidates pool.
  Every day in the candidate pool is compared with a pre-defined baseline imputed from surrounding days
  (e.g. the average of -7 and +7 days)
  and a score is generated to indicate deviation.
  The score is averaged if a holiday has multiple occurrences through the timeseries period.
  The holidays are ranked according to the magnitudes of the scores.
  Holidays are classified into:
  
      - model independently
      - model together
      - do not model
  
  according to their score magnitudes.
  For example, if the sum of the absolute scores is 1000,
  and the threshold for independent holidays is 0.8,
  the method keeps adding holidays to the independent modeling list
  from the largest magnitude until the sum reaches 1000 x 0.8 = 800.
  Then it continues to count the together modeling list.
  
  Attributes
  ----------
  baseline_offsets : `list` [`int`] or None
      The offsets in days to calculate baselines.
  post_search_days : `int` or None
      The number of days after each holiday to be counted as candidates.
  pre_search_days : `int` or None
      The number of days before each holiday to be counted as candidates.
  independent_holiday_thres : `float` or None
      A certain proportion of the total holiday effects that are allocated for holidays
      that are modeled independently. For example, 0.8 means the holidays that contribute
      to the first 80% of the holiday effects are modeled independently.
  together_holiday_thres : `float` or None
      A certain proportion of the total holiday effects that are allocated for holidays
      that are modeled together. For example, if ``independent_holiday_thres`` is 0.8 and
      ``together_holiday_thres`` is 0.9, then after the first 80% of the holiday effects
      are counted, the rest starts to be allocated for the holidays that are modeled together
      until the cum sum exceeds 0.9.
  extra_years : `int`, default 2
      Extra years after ``self.year_end`` to pull holidays in ``self.country_holiday_df``.
      This can be used to cover the forecast periods.
  df : `pandas.DataFrame` or None
      The timeseries after daily aggregation.
  time_col : `str` or None
      The column name for timestamps in ``df``.
  value_col : `str` or None
      The column name for values in ``df``.
  year_start : `int` or None
      The year of the first timeseries observation in ``df``.
  year_end : `int` or None
      The year of the last timeseries observation in ``df``.
  ts : `set` [`datetime`] or None
      The existing timestamps in ``df`` for fast look up.
  country_holiday_df : `pandas.DataFrame` or None
      The holidays between ``year_start`` and ``year_end``.
      This is the output from `pypi:holidays-ext`.
      Duplicates are dropped.
      Observed holidays are merged.
  all_holiday_dates : `list` [`datetime`] or None
      All holiday dates contained in ``country_holiday_df``.
  holidays : `list` [`str`] or None
      A list of holidays in ``country_holiday_df``.
  score_result : `dict` [`str`, `list` [`float`]] or None
      The scores from comparing holidays and their baselines.
      The keys are holidays.
      The values are a list of the scores for each occurrence.
  score_result_avg : `dict` [`str`, `float`] or None
      The scores from ``score_result`` where the values are averaged.
  result : `dict` [`str`, any]
      The output of the model. Includes:
  
          - "scores": `dict` [`str`, `list` [`float`]]
              The ``score_result`` from ``self._get_scores_for_holidays``.
          - "country_holiday_df": `pandas.DataFrame`
              The ``country_holiday_df`` from ``pypi:holidays_ext``.
          - "independent_holidays": `list` [`tuple` [`str`, `str`]]
              The holidays to be modeled independently. Each item is in (country, holiday) format.
          - "together_holidays_positive": `list` [`tuple` [`str`, `str`]]
              The holidays with positive effects to be modeled together. Each item is in (country, holiday) format.
          - "together_holidays_negative": `list` [`tuple` [`str`, `str`]]
              The holidays with negative effects to be modeled together. Each item is in (country, holiday) format.
          - "fig": `plotly.graph_objs.Figure`
              The visualization if activated.
class_docstr_tok: ["Implements","methods","to","automatically","infer","holiday","effects",".","The","class","works","for","daily","and","sub-daily","data",".","Sub-daily","data","is","aggregated","into","daily","data",".","It","pulls","holiday","candidates","from","`","pypi",":","holidays-ext","`",",","and","adds","a","pre-specified","number","of","days","before\/after","the","holiday","candidates","as","the","whole","holiday","candidates","pool",".","Every","day","in","the","candidate","pool","is","compared","with","a","pre-defined","baseline","imputed","from","surrounding","days","(","e.g",".","the","average","of","-7","and","+7","days",")","and","a","score","is","generated","to","indicate","deviation",".","The","score","is","averaged","if","a","holiday","has","multiple","occurrences","through","the","timeseries","period",".","The","holidays","are","ranked","according","to","the","magnitudes","of","the","scores",".","Holidays","are","classified","into",":","-","model","independently","-","model","together","-","do","not","model","according","to","their","score","magnitudes",".","For","example",",","if","the","sum","of","the","absolute","scores","is","1000",",","and","the","threshold","for","independent","holidays","is","0.8",",","the","method","keeps","adding","holidays","to","the","independent","modeling","list","from","the","largest","magnitude","until","the","sum","reaches","1000","x","0.8","=","800",".","Then","it","continues","to","count","the","together","modeling","list",".","Attributes","--","--","--","--","--","baseline_offsets",":","`","list","`","[","`","int","`","]","or","None","The","offsets","in","days","to","calculate","baselines",".","post_search_days",":","`","int","`","or","None","The","number","of","days","after","each","holiday","to","be","counted","as","candidates",".","pre_search_days",":","`","int","`","or","None","The","number","of","days","before","each","holiday","to","be","counted","as","candidates",".","independent_holiday_thres",":","`","float","`","or","None","A","certain","proportion","of","the","total","holiday","effects","that","are","allocated","for","holidays","that","are","modeled","independently",".","For","example",",","0.8","means","the","holidays","that","contribute","to","the","first","80","%","of","the","holiday","effects","are","modeled","independently",".","together_holiday_thres",":","`","float","`","or","None","A","certain","proportion","of","the","total","holiday","effects","that","are","allocated","for","holidays","that","are","modeled","together",".","For","example",",","if","``","independent_holiday_thres","``","is","0.8","and","``","together_holiday_thres","``","is","0.9",",","then","after","the","first","80","%","of","the","holiday","effects","are","counted",",","the","rest","starts","to","be","allocated","for","the","holidays","that","are","modeled","together","until","the","cum","sum","exceeds","0.9.","extra_years",":","`","int","`",",","default","2","Extra","years","after","``","self.year_end","``","to","pull","holidays","in","``","self.country_holiday_df","``",".","This","can","be","used","to","cover","the","forecast","periods",".","df",":","`","pandas.DataFrame","`","or","None","The","timeseries","after","daily","aggregation",".","time_col",":","`","str","`","or","None","The","column","name","for","timestamps","in","``","df","``",".","value_col",":","`","str","`","or","None","The","column","name","for","values","in","``","df","``",".","year_start",":","`","int","`","or","None","The","year","of","the","first","timeseries","observation","in","``","df","``",".","year_end",":","`","int","`","or","None","The","year","of","the","last","timeseries","observation","in","``","df","``",".","ts",":","`","set","`","[","`","datetime","`","]","or","None","The","existing","timestamps","in","``","df","``","for","fast","look","up",".","country_holiday_df",":","`","pandas.DataFrame","`","or","None","The","holidays","between","``","year_start","``","and","``","year_end","``",".","This","is","the","output","from","`","pypi",":","holidays-ext","`",".","Duplicates","are","dropped",".","Observed","holidays","are","merged",".","all_holiday_dates",":","`","list","`","[","`","datetime","`","]","or","None","All","holiday","dates","contained","in","``","country_holiday_df","``",".","holidays",":","`","list","`","[","`","str","`","]","or","None","A","list","of","holidays","in","``","country_holiday_df","``",".","score_result",":","`","dict","`","[","`","str","`",",","`","list","`","[","`","float","`","]","]","or","None","The","scores","from","comparing","holidays","and","their","baselines",".","The","keys","are","holidays",".","The","values","are","a","list","of","the","scores","for","each","occurrence",".","score_result_avg",":","`","dict","`","[","`","str","`",",","`","float","`","]","or","None","The","scores","from","``","score_result","``","where","the","values","are","averaged",".","result",":","`","dict","`","[","`","str","`",",","any","]","The","output","of","the","model",".","Includes",":","-","``","scores","''",":","`","dict","`","[","`","str","`",",","`","list","`","[","`","float","`","]","]","The","``","score_result","``","from","``","self._get_scores_for_holidays","``",".","-","``","country_holiday_df","''",":","`","pandas.DataFrame","`","The","``","country_holiday_df","``","from","``","pypi",":","holidays_ext","``",".","-","``","independent_holidays","''",":","`","list","`","[","`","tuple","`","[","`","str","`",",","`","str","`","]","]","The","holidays","to","be","modeled","independently",".","Each","item","is","in","(","country",",","holiday",")","format",".","-","``","together_holidays_positive","''",":","`","list","`","[","`","tuple","`","[","`","str","`",",","`","str","`","]","]","The","holidays","with","positive","effects","to","be","modeled","together",".","Each","item","is","in","(","country",",","holiday",")","format",".","-","``","together_holidays_negative","''",":","`","list","`","[","`","tuple","`","[","`","str","`",",","`","str","`","]","]","The","holidays","with","negative","effects","to","be","modeled","together",".","Each","item","is","in","(","country",",","holiday",")","format",".","-","``","fig","''",":","`","plotly.graph_objs.Figure","`","The","visualization","if","activated","."]
docstr: Gets the candidate holidays from a list of countries.
  Uses `pypi:holidays-ext`.
  Duplicates are dropped.
  Observed holidays are renamed to original holidays
  and corresponding original holidays in the same years are removed.
  
  Parameters
  ----------
  countries : `list` [`str`]
      A list of countries to look up candidate holidays.
  
  Returns
  -------
  result : `tuple`
      Includes:
  
          country_holiday_df : `pandas.DataFrame`
              The holidays between ``year_start`` and ``year_end``.
              This is the output from `pypi:holidays-ext`.
              Duplicates are dropped.
              Observed holidays are merged.
          holidays : `list` [`str`]
              A list of holidays in ``country_holiday_df``.
              The holidays are in the format of "{country_name}_{holiday_name}".
docstr_tok: ["Gets","the","candidate","holidays","from","a","list","of","countries",".","Uses","`","pypi",":","holidays-ext","`",".","Duplicates","are","dropped",".","Observed","holidays","are","renamed","to","original","holidays","and","corresponding","original","holidays","in","the","same","years","are","removed",".","Parameters","--","--","--","--","--","countries",":","`","list","`","[","`","str","`","]","A","list","of","countries","to","look","up","candidate","holidays",".","Returns","--","--","--","-","result",":","`","tuple","`","Includes",":","country_holiday_df",":","`","pandas.DataFrame","`","The","holidays","between","``","year_start","``","and","``","year_end","``",".","This","is","the","output","from","`","pypi",":","holidays-ext","`",".","Duplicates","are","dropped",".","Observed","holidays","are","merged",".","holidays",":","`","list","`","[","`","str","`","]","A","list","of","holidays","in","``","country_holiday_df","``",".","The","holidays","are","in","the","format","of","``","{","country_name","}","_","{","holiday_name","}","''","."]
code: def _get_candidate_holidays(
      self, countries: List[str]
  ) -> (pd.DataFrame, List[str]):
      """Gets the candidate holidays from a list of countries.
      Uses `pypi:holidays-ext`.
      Duplicates are dropped.
      Observed holidays are renamed to original holidays
      and corresponding original holidays in the same years are removed.
  
      Parameters
      ----------
      countries : `list` [`str`]
          A list of countries to look up candidate holidays.
  
      Returns
      -------
      result : `tuple`
          Includes:
  
              country_holiday_df : `pandas.DataFrame`
                  The holidays between ``year_start`` and ``year_end``.
                  This is the output from `pypi:holidays-ext`.
                  Duplicates are dropped.
                  Observed holidays are merged.
              holidays : `list` [`str`]
                  A list of holidays in ``country_holiday_df``.
                  The holidays are in the format of "{country_name}_{holiday_name}".
  
      """
      country_holiday_df = get_holiday_df(
          country_list=countries,
          years=list(
              range(self.year_start, self.year_end + self.extra_years)
          ),
      )
      # Drops duplications.
      country_holiday_df.drop_duplicates(
          keep="first", subset=["ts"], inplace=True
      )
  
      # Handles observed holidays.
      # If observed holiday and original holiday are both listed in the same year,
      # the observed holiday will be renamed to the original holiday
      # and the original holiday in the same year will be removed.
  
      # Sub-df that contains observed holidays only.
      observed_df = country_holiday_df[
          country_holiday_df["holiday"].str[-10:] == "(Observed)"
      ]
      # Row indices to rename.
      rows_to_rename = observed_df.index.tolist()
      # Date-holiday tuple to remove.
      # ":-11" truncates the " (Observed)" suffix.
      # This is used to identify rows to remove.
      date_holiday_to_remove = [
          (row[1]["ts"], row[1]["holiday"][:-11])
          for row in observed_df.iterrows()
      ]
      # Row indices to remove.
      # For each (date, holiday) tuple, look up the match in ``country_holiday_df`` and record the row indices.
      # The match happens when the holiday name matches and the time diff is at most 3 days.
      rows_to_remove = [
          idx
          for date, holiday in date_holiday_to_remove
          for idx in country_holiday_df[
              (
                  abs(
                      (
                          pd.DatetimeIndex(country_holiday_df["ts"])
                          - date
                      ).days
                  )
                  <= 3
              )
              & (country_holiday_df["holiday"] == holiday)
          ].index.tolist()
      ]
      # Renames and removes.
      country_holiday_df.loc[
          rows_to_rename, "holiday"
      ] = country_holiday_df.loc[rows_to_rename, "holiday"].str[:-11]
      country_holiday_df.loc[
          rows_to_rename, "country_holiday"
      ] = country_holiday_df.loc[rows_to_rename, "country_holiday"].str[
          :-11
      ]
      country_holiday_df.drop(rows_to_remove, axis=0, inplace=True)
      country_holiday_df.reset_index(drop=True, inplace=True)
  
      holidays = country_holiday_df["country_holiday"].unique().tolist()
      return country_holiday_df, holidays
  
imports: import datetime
  from datetime import timedelta
  from typing import Dict
  from typing import List
  from typing import Optional
  from typing import Set
  from typing import Tuple
  from typing import Union
  import numpy
  import pandas
  from holidays_ext.get_holidays import get_holiday_df
  from plotly import graph_objs
  from plotly.subplots import make_subplots
  from greykite.common.constants import EVENT_DF_DATE_COL
  from greykite.common.constants import EVENT_DF_LABEL_COL
  from greykite.common.constants import EVENT_INDICATOR
  from greykite.common.constants import TIME_COL
  from greykite.common.constants import VALUE_COL
  from greykite.common.logging import LoggingLevelEnum
  from greykite.common.logging import log_message
label_desc: Use image node_id 4 for calling the HolidayInferrer obj's underlying member method code with example usage: obj._get_candidate_holidays(countries) and returns: country_holiday_df,  holidays