class_docstr: Incrementally search for hyper-parameters on models that support partial_fit
  
  This incremental hyper-parameter optimization class starts training the
  model on many hyper-parameters on a small amount of data, and then only
  continues training those models that seem to be performing well.
  
  This class will decay the number of parameters over time. At time step
  ``k``, this class will retain ``1 / (k + 1)`` fraction of the highest
  performing models.
  
  Parameters
  ----------
  estimator : estimator object.
      A object of that type is instantiated for each initial hyperparameter
      combination. This is assumed to implement the scikit-learn estimator
      interface. Either estimator needs to provide a `score`` function,
      or ``scoring`` must be passed. The estimator must implement
      ``partial_fit``, ``set_params``, and work well with ``clone``.
  
  parameters : dict
      Dictionary with parameters names (string) as keys and distributions
      or lists of parameters to try. Distributions must provide a ``rvs``
      method for sampling (such as those from scipy.stats.distributions).
      If a list is given, it is sampled uniformly.
  
  n_initial_parameters : int, default=10
      Number of parameter settings that are sampled.
      This trades off runtime vs quality of the solution.
  
      Alternatively, you can set this to ``"grid"`` to do a full grid search.
  
  patience : int, default False
      If specified, training stops when the score does not increase by
      ``tol`` after ``patience`` calls to ``partial_fit``. Off by default.
  
  fits_per_scores : int, optional, default=1
      If ``patience`` is used the maximum number of ``partial_fit`` calls
      between ``score`` calls.
  
  scores_per_fit : int, default 1
      If ``patience`` is used the maximum number of ``partial_fit`` calls
      between ``score`` calls.
  
  tol : float, default 0.001
      The required level of improvement to consider stopping training on
      that model. The most recent score must be at at most ``tol`` better
      than the all of the previous ``patience`` scores for that model.
      Increasing ``tol`` will tend to reduce training time, at the cost
      of worse models.
  
  max_iter : int, default 100
      Maximum number of partial fit calls per model.
  
  test_size : float
      Fraction of the dataset to hold out for computing test scores.
      Defaults to the size of a single partition of the input training set
  
      .. note::
  
         The training dataset should fit in memory on a single machine.
         Adjust the ``test_size`` parameter as necessary to achieve this.
  
  random_state : int, RandomState instance or None, optional, default: None
      If int, random_state is the seed used by the random number generator;
      If RandomState instance, random_state is the random number generator;
      If None, the random number generator is the RandomState instance used
      by `np.random`.
  
  scoring : string, callable, list/tuple, dict or None, default: None
      A single string (see :ref:`scoring_parameter`) or a callable
      (see :ref:`scoring`) to evaluate the predictions on the test set.
  
      For evaluating multiple metrics, either give a list of (unique) strings
      or a dict with names as keys and callables as values.
  
      NOTE that when using custom scorers, each scorer should return a single
      value. Metric functions returning a list/array of values can be wrapped
      into multiple scorers that return one value each.
  
      See :ref:`multimetric_grid_search` for an example.
  
      If None, the estimator's default scorer (if available) is used.
  
  verbose : bool, float, int, optional, default: False
      If False (default), don't print logs (or pipe them to stdout). However,
      standard logging will still be used.
  
      If True, print logs and use standard logging.
  
      If float, print/log approximately ``verbose`` fraction of the time.
  
  prefix : str, optional, default=""
      While logging, add ``prefix`` to each message.
  
  decay_rate : float, default 1.0
      How quickly to decrease the number partial future fit calls.
      Higher `decay_rate` will result in lower training times, at the cost
      of worse models.
  
      The default ``decay_rate=1.0`` is chosen because it has some theoretical
      motivation [1]_.
  
  Attributes
  ----------
  cv_results_ : dict of np.ndarrays
      This dictionary has keys
  
      * ``mean_partial_fit_time``
      * ``mean_score_time``
      * ``std_partial_fit_time``
      * ``std_score_time``
      * ``test_score``
      * ``rank_test_score``
      * ``model_id``
      * ``partial_fit_calls``
      * ``params``
      * ``param_{key}``, where ``key`` is every key in ``params``.
  
      The values in the ``test_score`` key correspond to the last score a model
      received on the hold out dataset. The key ``model_id`` corresponds with
      ``history_``. This dictionary can be imported into Pandas.
  
  model_history_ : dict of lists of dict
      A dictionary of each models history. This is a reorganization of
      ``history_``: the same information is present but organized per model.
  
      This data has the structure  ``{model_id: hist}`` where ``hist`` is a
      subset of ``history_`` and ``model_id`` are model identifiers.
  
  history_ : list of dicts
      Information about each model after each ``partial_fit`` call. Each dict
      the keys
  
      * ``partial_fit_time``
      * ``score_time``
      * ``score``
      * ``model_id``
      * ``params``
      * ``partial_fit_calls``
      * ``elapsed_wall_time``
  
      The key ``model_id`` corresponds to the ``model_id`` in ``cv_results_``.
      This list of dicts can be imported into Pandas.
  
  best_estimator_ : BaseEstimator
      The model with the highest validation score among all the models
      retained by the "inverse decay" algorithm.
  
  best_score_ : float
      Score achieved by ``best_estimator_`` on the validation set after the
      final call to ``partial_fit``.
  
  best_index_ : int
      Index indicating which estimator in ``cv_results_`` corresponds to
      the highest score.
  
  best_params_ : dict
      Dictionary of best parameters found on the hold-out data.
  
  scorer_ :
      The function used to score models, which has a call signature of
      ``scorer_(estimator, X, y)``.
  
  n_splits_ : int
      Number of cross validation splits.
  
  multimetric_ : bool
      Whether this cross validation search uses multiple metrics.
  
  Notes
  -----
  When ``decay_rate==1``, this class approximates the
  number of ``partial_fit`` calls that :class:`SuccessiveHalvingSearchCV`
  performs. If ``n_initial_parameters`` is configured properly with
  ``decay_rate=1``, it's possible this class will mirror the most aggressive
  bracket of :class:`HyperbandSearchCV`. This might yield good results
  and/or find good models, but is untested.
  
  References
  ----------
  .. [1] Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A.
         (2017). Hyperband: A novel bandit-based approach to hyperparameter
         optimization. The Journal of Machine Learning Research, 18(1),
         6765-6816. http://www.jmlr.org/papers/volume18/16-558/16-558.pdf
class_docstr_tok: ["Incrementally","search","for","hyper-parameters","on","models","that","support","partial_fit","This","incremental","hyper-parameter","optimization","class","starts","training","the","model","on","many","hyper-parameters","on","a","small","amount","of","data",",","and","then","only","continues","training","those","models","that","seem","to","be","performing","well",".","This","class","will","decay","the","number","of","parameters","over","time",".","At","time","step","``","k","``",",","this","class","will","retain","``","1","\/","(","k","+","1",")","``","fraction","of","the","highest","performing","models",".","Parameters","--","--","--","--","--","estimator",":","estimator","object",".","A","object","of","that","type","is","instantiated","for","each","initial","hyperparameter","combination",".","This","is","assumed","to","implement","the","scikit-learn","estimator","interface",".","Either","estimator","needs","to","provide","a","`","score","``","function",",","or","``","scoring","``","must","be","passed",".","The","estimator","must","implement","``","partial_fit","``",",","``","set_params","``",",","and","work","well","with","``","clone","``",".","parameters",":","dict","Dictionary","with","parameters","names","(","string",")","as","keys","and","distributions","or","lists","of","parameters","to","try",".","Distributions","must","provide","a","``","rvs","``","method","for","sampling","(","such","as","those","from","scipy.stats.distributions",")",".","If","a","list","is","given",",","it","is","sampled","uniformly",".","n_initial_parameters",":","int",",","default=10","Number","of","parameter","settings","that","are","sampled",".","This","trades","off","runtime","vs","quality","of","the","solution",".","Alternatively",",","you","can","set","this","to","``","``","grid","''","``","to","do","a","full","grid","search",".","patience",":","int",",","default","False","If","specified",",","training","stops","when","the","score","does","not","increase","by","``","tol","``","after","``","patience","``","calls","to","``","partial_fit","``",".","Off","by","default",".","fits_per_scores",":","int",",","optional",",","default=1","If","``","patience","``","is","used","the","maximum","number","of","``","partial_fit","``","calls","between","``","score","``","calls",".","scores_per_fit",":","int",",","default","1","If","``","patience","``","is","used","the","maximum","number","of","``","partial_fit","``","calls","between","``","score","``","calls",".","tol",":","float",",","default","0.001","The","required","level","of","improvement","to","consider","stopping","training","on","that","model",".","The","most","recent","score","must","be","at","at","most","``","tol","``","better","than","the","all","of","the","previous","``","patience","``","scores","for","that","model",".","Increasing","``","tol","``","will","tend","to","reduce","training","time",",","at","the","cost","of","worse","models",".","max_iter",":","int",",","default","100","Maximum","number","of","partial","fit","calls","per","model",".","test_size",":","float","Fraction","of","the","dataset","to","hold","out","for","computing","test","scores",".","Defaults","to","the","size","of","a","single","partition","of","the","input","training","set","..","note",":",":","The","training","dataset","should","fit","in","memory","on","a","single","machine",".","Adjust","the","``","test_size","``","parameter","as","necessary","to","achieve","this",".","random_state",":","int",",","RandomState","instance","or","None",",","optional",",","default",":","None","If","int",",","random_state","is","the","seed","used","by","the","random","number","generator",";","If","RandomState","instance",",","random_state","is","the","random","number","generator",";","If","None",",","the","random","number","generator","is","the","RandomState","instance","used","by","`","np.random","`",".","scoring",":","string",",","callable",",","list\/tuple",",","dict","or","None",",","default",":","None","A","single","string","(","see",":","ref",":","`","scoring_parameter","`",")","or","a","callable","(","see",":","ref",":","`","scoring","`",")","to","evaluate","the","predictions","on","the","test","set",".","For","evaluating","multiple","metrics",",","either","give","a","list","of","(","unique",")","strings","or","a","dict","with","names","as","keys","and","callables","as","values",".","NOTE","that","when","using","custom","scorers",",","each","scorer","should","return","a","single","value",".","Metric","functions","returning","a","list\/array","of","values","can","be","wrapped","into","multiple","scorers","that","return","one","value","each",".","See",":","ref",":","`","multimetric_grid_search","`","for","an","example",".","If","None",",","the","estimator","'s","default","scorer","(","if","available",")","is","used",".","verbose",":","bool",",","float",",","int",",","optional",",","default",":","False","If","False","(","default",")",",","do","n't","print","logs","(","or","pipe","them","to","stdout",")",".","However",",","standard","logging","will","still","be","used",".","If","True",",","print","logs","and","use","standard","logging",".","If","float",",","print\/log","approximately","``","verbose","``","fraction","of","the","time",".","prefix",":","str",",","optional",",","default=","''","''","While","logging",",","add","``","prefix","``","to","each","message",".","decay_rate",":","float",",","default","1.0","How","quickly","to","decrease","the","number","partial","future","fit","calls",".","Higher","`","decay_rate","`","will","result","in","lower","training","times",",","at","the","cost","of","worse","models",".","The","default","``","decay_rate=1.0","``","is","chosen","because","it","has","some","theoretical","motivation","[","1","]","_",".","Attributes","--","--","--","--","--","cv_results_",":","dict","of","np.ndarrays","This","dictionary","has","keys","*","``","mean_partial_fit_time","``","*","``","mean_score_time","``","*","``","std_partial_fit_time","``","*","``","std_score_time","``","*","``","test_score","``","*","``","rank_test_score","``","*","``","model_id","``","*","``","partial_fit_calls","``","*","``","params","``","*","``","param_","{","key","}","``",",","where","``","key","``","is","every","key","in","``","params","``",".","The","values","in","the","``","test_score","``","key","correspond","to","the","last","score","a","model","received","on","the","hold","out","dataset",".","The","key","``","model_id","``","corresponds","with","``","history_","``",".","This","dictionary","can","be","imported","into","Pandas",".","model_history_",":","dict","of","lists","of","dict","A","dictionary","of","each","models","history",".","This","is","a","reorganization","of","``","history_","``",":","the","same","information","is","present","but","organized","per","model",".","This","data","has","the","structure","``","{","model_id",":","hist","}","``","where","``","hist","``","is","a","subset","of","``","history_","``","and","``","model_id","``","are","model","identifiers",".","history_",":","list","of","dicts","Information","about","each","model","after","each","``","partial_fit","``","call",".","Each","dict","the","keys","*","``","partial_fit_time","``","*","``","score_time","``","*","``","score","``","*","``","model_id","``","*","``","params","``","*","``","partial_fit_calls","``","*","``","elapsed_wall_time","``","The","key","``","model_id","``","corresponds","to","the","``","model_id","``","in","``","cv_results_","``",".","This","list","of","dicts","can","be","imported","into","Pandas",".","best_estimator_",":","BaseEstimator","The","model","with","the","highest","validation","score","among","all","the","models","retained","by","the","``","inverse","decay","''","algorithm",".","best_score_",":","float","Score","achieved","by","``","best_estimator_","``","on","the","validation","set","after","the","final","call","to","``","partial_fit","``",".","best_index_",":","int","Index","indicating","which","estimator","in","``","cv_results_","``","corresponds","to","the","highest","score",".","best_params_",":","dict","Dictionary","of","best","parameters","found","on","the","hold-out","data",".","scorer_",":","The","function","used","to","score","models",",","which","has","a","call","signature","of","``","scorer_","(","estimator",",","X",",","y",")","``",".","n_splits_",":","int","Number","of","cross","validation","splits",".","multimetric_",":","bool","Whether","this","cross","validation","search","uses","multiple","metrics",".","Notes","--","--","-","When","``","decay_rate==1","``",",","this","class","approximates","the","number","of","``","partial_fit","``","calls","that",":","class",":","`","SuccessiveHalvingSearchCV","`","performs",".","If","``","n_initial_parameters","``","is","configured","properly","with","``","decay_rate=1","``",",","it","'s","possible","this","class","will","mirror","the","most","aggressive","bracket","of",":","class",":","`","HyperbandSearchCV","`",".","This","might","yield","good","results","and\/or","find","good","models",",","but","is","untested",".","References","--","--","--","--","--","..","[","1","]","Li",",","L.",",","Jamieson",",","K.",",","DeSalvo",",","G.",",","Rostamizadeh",",","A.",",","&","Talwalkar",",","A",".","(","2017",")",".","Hyperband",":","A","novel","bandit-based","approach","to","hyperparameter","optimization",".","The","Journal","of","Machine","Learning","Research",",","18","(","1",")",",","6765-6816.","http",":","\/\/www.jmlr.org\/papers\/volume18\/16-558\/16-558.pdf"]
code: def _adapt(self, info):
      # First, have an adaptive algorithm
      if self.n_initial_parameters == "grid":
          start = len(ParameterGrid(self.parameters))
      else:
          start = self.n_initial_parameters
  
      def inverse(time):
          """Decrease target number of models inversely with time"""
          return int(start / (1 + time) ** self.decay_rate)
  
      example = toolz.first(info.values())
      time_step = example[-1]["partial_fit_calls"]
  
      current_time_step = time_step + 1
      next_time_step = current_time_step
  
      if inverse(current_time_step) == 0:
          # we'll never get out of here
          next_time_step = 1
  
      while inverse(current_time_step) == inverse(next_time_step) and (
          self.decay_rate
          and not self.patience
          or next_time_step - current_time_step < self.fits_per_score
      ):
          next_time_step += 1
  
      target = max(1, inverse(next_time_step))
      best = toolz.topk(
          target, info, key=lambda k: info[k][-1]["score"]
      )
  
      if len(best) == 1:
          [best] = best
          return {best: 0}
      steps = next_time_step - current_time_step
      instructions = {b: steps for b in best}
      return instructions
  
imports: from __future__ import division
  import itertools
  import logging
  import operator
  import sys
  from collections import defaultdict, namedtuple
  from copy import deepcopy
  from time import time
  from typing import Any, Callable, Dict, List, Optional, Tuple, Union
  from warnings import warn
  import dask
  import dask.array
  import dask.dataframe
  import numpy
  import scipy.stats
  import toolz
  from dask.distributed import Future, default_client, futures_of, wait
  from distributed.utils import log_errors
  from sklearn.base import BaseEstimator, clone
  from sklearn.linear_model import SGDClassifier
  from sklearn.metrics import check_scoring
  from sklearn.model_selection import ParameterGrid, ParameterSampler
  from sklearn.utils import check_random_state
  from sklearn.utils.metaestimators import available_if
  from sklearn.utils.validation import check_is_fitted
  from .._compat import DISTRIBUTED_2021_02_0, annotate, dummy_context
  from .._typing import ArrayLike, Int
  from .._utils import LoggingContext
  from ..wrappers import ParallelPostFit
  from ._split import train_test_split
  from .utils import estimator_has
label_desc: Use image node_id 3 for calling the InverseDecaySearchCV obj's underlying member method code with example usage: obj._adapt(info) and returns: instructions, int, dict