class_docstr: Ultra-high, conditional sure independence screening
  
  This is an adjusted version of Fan's sure independence screening.
  
  Parameters
  ----------
  model : instance of penalizing model
      examples: GLMPenalized, PoissonPenalized and LogitPenalized.
      The attributes of the model instance `pen_weight` and `penal` will be
      ignored.
  pen_weight : None or float
      penalization weight use in SCAD penalized MLE
  k_add : int
      number of exog to add during expansion or forward selection
      see Notes section for tie handling
  k_max_add : int
      maximum number of variables to include during variable addition, i.e.
      forward selection. default is 30
  threshold_trim : float
      threshold for trimming parameters to zero, default is 1e-4
  k_max_included : int
      maximum total number of variables to include in model.
  ranking_attr : str
      This determines the result attribute or model method that is used for
      the ranking of exog to include. The availability of attributes depends
      on the model.
      Default is 'resid_pearson', 'model.score_factor' can be used in GLM.
  ranking_project : bool
      If ranking_project is True, then the exog candidates for inclusion are
      first projected on the already included exog before the computation
      of the ranking measure. This brings the ranking measure closer to
      the statistic of a score test for variable addition.
  
  Notes
  -----
  Status: experimental, tested only on a limited set of models and
  with a limited set of model options.
  
  Tie handling: If there are ties at the decision threshold, then all those
  tied exog columns are treated in the same way. During forward selection
  all exog columns with the same boundary value are included. During
  elimination, the tied columns are not dropped. Consequently, if ties are
  present, then the number of included exog can be larger than specified
  by k_add, k_max_add and k_max_included.
  
  The screening algorithm works similar to step wise regression. Each
  iteration of the screening algorithm includes a forward selection step
  where variables are added to the model, and a backwards selection step
  where variables are removed. In contrast to step wise regression, we add
  a fixed number of variables at each forward selection step. The
  backwards selection step is based on SCAD penalized estimation and
  trimming of variables with estimated coefficients below a threshold.
  The tuning parameters can be used to adjust the number of variables to add
  and to include depending on the size of the dataset.
  
  There is currently no automatic tuning parameter selection. Candidate
  explanatory variables should be standardized or should be on a similar
  scale because penalization and trimming are based on the absolute values
  of the parameters.
  
  
  TODOs and current limitations:
  
  freq_weights are not supported in this. Candidate ranking uses
  moment condition with resid_pearson or others without freq_weights.
  pearson_resid: GLM resid_pearson does not include freq_weights.
  
  variable names: do we keep track of those? currently made-up names
  
  currently only supports numpy arrays, no exog type check or conversion
  
  currently only single columns are selected, no terms (multi column exog)
class_docstr_tok: ["Ultra-high",",","conditional","sure","independence","screening","This","is","an","adjusted","version","of","Fan","'s","sure","independence","screening",".","Parameters","--","--","--","--","--","model",":","instance","of","penalizing","model","examples",":","GLMPenalized",",","PoissonPenalized","and","LogitPenalized",".","The","attributes","of","the","model","instance","`","pen_weight","`","and","`","penal","`","will","be","ignored",".","pen_weight",":","None","or","float","penalization","weight","use","in","SCAD","penalized","MLE","k_add",":","int","number","of","exog","to","add","during","expansion","or","forward","selection","see","Notes","section","for","tie","handling","k_max_add",":","int","maximum","number","of","variables","to","include","during","variable","addition",",","i.e",".","forward","selection",".","default","is","30","threshold_trim",":","float","threshold","for","trimming","parameters","to","zero",",","default","is","1e-4","k_max_included",":","int","maximum","total","number","of","variables","to","include","in","model",".","ranking_attr",":","str","This","determines","the","result","attribute","or","model","method","that","is","used","for","the","ranking","of","exog","to","include",".","The","availability","of","attributes","depends","on","the","model",".","Default","is","'resid_pearson","'",",","'model.score_factor","'","can","be","used","in","GLM",".","ranking_project",":","bool","If","ranking_project","is","True",",","then","the","exog","candidates","for","inclusion","are","first","projected","on","the","already","included","exog","before","the","computation","of","the","ranking","measure",".","This","brings","the","ranking","measure","closer","to","the","statistic","of","a","score","test","for","variable","addition",".","Notes","--","--","-","Status",":","experimental",",","tested","only","on","a","limited","set","of","models","and","with","a","limited","set","of","model","options",".","Tie","handling",":","If","there","are","ties","at","the","decision","threshold",",","then","all","those","tied","exog","columns","are","treated","in","the","same","way",".","During","forward","selection","all","exog","columns","with","the","same","boundary","value","are","included",".","During","elimination",",","the","tied","columns","are","not","dropped",".","Consequently",",","if","ties","are","present",",","then","the","number","of","included","exog","can","be","larger","than","specified","by","k_add",",","k_max_add","and","k_max_included",".","The","screening","algorithm","works","similar","to","step","wise","regression",".","Each","iteration","of","the","screening","algorithm","includes","a","forward","selection","step","where","variables","are","added","to","the","model",",","and","a","backwards","selection","step","where","variables","are","removed",".","In","contrast","to","step","wise","regression",",","we","add","a","fixed","number","of","variables","at","each","forward","selection","step",".","The","backwards","selection","step","is","based","on","SCAD","penalized","estimation","and","trimming","of","variables","with","estimated","coefficients","below","a","threshold",".","The","tuning","parameters","can","be","used","to","adjust","the","number","of","variables","to","add","and","to","include","depending","on","the","size","of","the","dataset",".","There","is","currently","no","automatic","tuning","parameter","selection",".","Candidate","explanatory","variables","should","be","standardized","or","should","be","on","a","similar","scale","because","penalization","and","trimming","are","based","on","the","absolute","values","of","the","parameters",".","TODOs","and","current","limitations",":","freq_weights","are","not","supported","in","this",".","Candidate","ranking","uses","moment","condition","with","resid_pearson","or","others","without","freq_weights",".","pearson_resid",":","GLM","resid_pearson","does","not","include","freq_weights",".","variable","names",":","do","we","keep","track","of","those","?","currently","made-up","names","currently","only","supports","numpy","arrays",",","no","exog","type","check","or","conversion","currently","only","single","columns","are","selected",",","no","terms","(","multi","column","exog",")"]
code: def __init__(
      self,
      model,
      pen_weight=None,
      use_weights=True,
      k_add=30,
      k_max_add=30,
      threshold_trim=1e-4,
      k_max_included=20,
      ranking_attr="resid_pearson",
      ranking_project=True,
  ):
      self.model = model
      self.model_class = model.__class__
      self.init_kwds = model._get_init_kwds()
      # pen_weight and penal are explicitly included
      # TODO: check what we want to do here
      self.init_kwds.pop("pen_weight", None)
      self.init_kwds.pop("penal", None)
  
      self.endog = model.endog
      self.exog_keep = model.exog
      self.k_keep = model.exog.shape[1]
      self.nobs = len(self.endog)
      self.penal = self._get_penal()
  
      if pen_weight is not None:
          self.pen_weight = pen_weight
      else:
          self.pen_weight = self.nobs * 10
  
      # option for screening algorithm
      self.use_weights = use_weights
      self.k_add = k_add
      self.k_max_add = k_max_add
      self.threshold_trim = threshold_trim
      self.k_max_included = k_max_included
      self.ranking_attr = ranking_attr
      self.ranking_project = ranking_project
  
file_path: statsmodels/statsmodels/base/_screening.py
imports: from collections import defaultdict
  import numpy
  from statsmodels.base._penalties import SCADSmoothed
label_desc: Use image node_id 1 to create a new VariableScreening object  with example: obj = VariableScreening(model, pen_weight, use_weights, k_add, k_max_add, threshold_trim, k_max_included, ranking_attr, ranking_project)