# dask_ml.model_selection.InverseDecaySearchCV¶

class dask_ml.model_selection.InverseDecaySearchCV(estimator, parameters, n_initial_parameters=10, test_size=None, patience=False, tol=0.001, fits_per_score=1, max_iter=100, random_state=None, scoring=None, verbose=False, prefix='', decay_rate=1.0)

Incrementally search for hyper-parameters on models that support partial_fit

This incremental hyper-parameter optimization class starts training the model on many hyper-parameters on a small amount of data, and then only continues training those models that seem to be performing well.

This class will decay the number of parameters over time. At time step k, this class will retain 1 / (k + 1) fraction of the highest performing models.

Parameters: estimator : estimator object. A object of that type is instantiated for each initial hyperparameter combination. This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed. The estimator must implement partial_fit, set_params, and work well with clone. parameters : dict Dictionary with parameters names (string) as keys and distributions or lists of parameters to try. Distributions must provide a rvs method for sampling (such as those from scipy.stats.distributions). If a list is given, it is sampled uniformly. n_initial_parameters : int, default=10 Number of parameter settings that are sampled. This trades off runtime vs quality of the solution. Alternatively, you can set this to "grid" to do a full grid search. patience : int, default False If specified, training stops when the score does not increase by tol after patience calls to partial_fit. Off by default. fits_per_scores : int, optional, default=1 If patience is used the maximum number of partial_fit calls between score calls. scores_per_fit : int, default 1 If patience is used the maximum number of partial_fit calls between score calls. tol : float, default 0.001 The required level of improvement to consider stopping training on that model. The most recent score must be at at most tol better than the all of the previous patience scores for that model. Increasing tol will tend to reduce training time, at the cost of worse models. max_iter : int, default 100 Maximum number of partial fit calls per model. test_size : float Fraction of the dataset to hold out for computing test scores. Defaults to the size of a single partition of the input training set Note The training dataset should fit in memory on a single machine. Adjust the test_size parameter as necessary to achieve this. random_state : int, RandomState instance or None, optional, default: None If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random. scoring : string, callable, list/tuple, dict or None, default: None A single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions) to evaluate the predictions on the test set. For evaluating multiple metrics, either give a list of (unique) strings or a dict with names as keys and callables as values. NOTE that when using custom scorers, each scorer should return a single value. Metric functions returning a list/array of values can be wrapped into multiple scorers that return one value each. See Specifying multiple metrics for evaluation for an example. If None, the estimator’s default scorer (if available) is used. verbose : bool, float, int, optional, default: False If False (default), don’t print logs (or pipe them to stdout). However, standard logging will still be used. If True, print logs and use standard logging. If float, print/log approximately verbose fraction of the time. prefix : str, optional, default=”” While logging, add prefix to each message. decay_rate : float, default 1.0 How quickly to decrease the number partial future fit calls. Higher decay_rate will result in lower training times, at the cost of worse models. The default decay_rate=1.0 is chosen because it has some theoritical motivation . cv_results_ : dict of np.ndarrays This dictionary has keys mean_partial_fit_time mean_score_time std_partial_fit_time std_score_time test_score rank_test_score model_id partial_fit_calls params param_{key}, where key is every key in params. The values in the test_score key correspond to the last score a model received on the hold out dataset. The key model_id corresponds with history_. This dictionary can be imported into Pandas. model_history_ : dict of lists of dict A dictionary of each models history. This is a reorganization of history_: the same information is present but organized per model. This data has the structure {model_id: hist} where hist is a subset of history_ and model_id are model identifiers. history_ : list of dicts Information about each model after each partial_fit call. Each dict the keys partial_fit_time score_time score model_id params partial_fit_calls elapsed_wall_time The key model_id corresponds to the model_id in cv_results_. This list of dicts can be imported into Pandas. best_estimator_ : BaseEstimator The model with the highest validation score among all the models retained by the “inverse decay” algorithm. best_score_ : float Score achieved by best_estimator_ on the vaidation set after the final call to partial_fit. best_index_ : int Index indicating which estimator in cv_results_ corresponds to the highest score. best_params_ : dict Dictionary of best parameters found on the hold-out data. scorer_ : The function used to score models, which has a call signature of scorer_(estimator, X, y). n_splits_ : int Number of cross validation splits. multimetric_ : bool Whether this cross validation search uses multiple metrics.

Notes

When decay_rate==1, this class approximates the number of partial_fit calls that SuccesiveHalvingSearchCV performs. If n_initial_parameters is configured properly with decay_rate=1, it’s possible this class will mirror the most aggressive bracket of HyperbandSearchCV. This might yield good results and/or find good models, but is untested.

References

  Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2017). Hyperband: A novel bandit-based approach to hyperparameter optimization. The Journal of Machine Learning Research, 18(1), 6765-6816. http://www.jmlr.org/papers/volume18/16-558/16-558.pdf

Methods

 decision_function(X) fit(X[, y]) Find the best parameters for a particular model. get_params([deep]) Get parameters for this estimator. inverse_transform(Xt) predict(X) Predict for X. predict_log_proba(X) Log of proability estimates. predict_proba(X) Probability estimates. score(X[, y]) Returns the score on the given data. set_params(**params) Set the parameters of this estimator. transform(X)
 partial_fit
__init__(estimator, parameters, n_initial_parameters=10, test_size=None, patience=False, tol=0.001, fits_per_score=1, max_iter=100, random_state=None, scoring=None, verbose=False, prefix='', decay_rate=1.0)

Initialize self. See help(type(self)) for accurate signature.

fit(X, y=None, **fit_params)

Find the best parameters for a particular model.

Parameters: X, y : array-like **fit_params Additional partial fit keyword arguments for the estimator.
get_params(deep=True)

Get parameters for this estimator.

Parameters: deep : bool, default=True If True, will return the parameters for this estimator and contained subobjects that are estimators. params : mapping of string to any Parameter names mapped to their values.
predict(X)

Predict for X.

For dask inputs, a dask array or dataframe is returned. For other inputs (NumPy array, pandas dataframe, scipy sparse matrix), the regular return value is returned.

Parameters: X : array-like y : array-like
predict_log_proba(X)

Log of proability estimates.

For dask inputs, a dask array or dataframe is returned. For other inputs (NumPy array, pandas dataframe, scipy sparse matrix), the regular return value is returned.

If the underlying estimator does not have a predict_proba method, then an AttributeError is raised.

Parameters: X : array or dataframe y : array-like
predict_proba(X)

Probability estimates.

For dask inputs, a dask array or dataframe is returned. For other inputs (NumPy array, pandas dataframe, scipy sparse matrix), the regular return value is returned.

If the underlying estimator does not have a predict_proba method, then an AttributeError is raised.

Parameters: X : array or dataframe y : array-like
score(X, y=None) → float

Returns the score on the given data.

Parameters: X : array-like, shape = [n_samples, n_features] Input data, where n_samples is the number of samples and n_features is the number of features. y : array-like, shape = [n_samples] or [n_samples, n_output], optional Target relative to X for classification or regression; None for unsupervised learning. score : float return self.estimator.score(X, y)
set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter>` so that it’s possible to update each component of a nested object.

Parameters: **params : dict Estimator parameters. self : object Estimator instance.
{"ft_lang_label":"__label__en","ft_lang_prob":0.6073777,"math_prob":0.9350256,"size":9538,"snap":"2020-34-2020-40","text_gpt3_token_len":2249,"char_repetition_ratio":0.1320537,"word_repetition_ratio":0.12381646,"special_character_ratio":0.22226882,"punctuation_ratio":0.17643575,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9890563,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-09-30T10:11:51Z\",\"WARC-Record-ID\":\"<urn:uuid:4567a9d0-5dc4-440e-987a-5987547aca63>\",\"Content-Length\":\"48203\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:8283a89c-b8ea-4dc0-8a0b-ae105e8bfb0f>\",\"WARC-Concurrent-To\":\"<urn:uuid:041b9c27-c90c-460b-ac9d-c2ff9ec813c8>\",\"WARC-IP-Address\":\"104.18.44.212\",\"WARC-Target-URI\":\"https://ml.dask.org/modules/generated/dask_ml.model_selection.InverseDecaySearchCV.html\",\"WARC-Payload-Digest\":\"sha1:JSUFHZTQEB7SIKJZECSWM5P2HVZDRJYH\",\"WARC-Block-Digest\":\"sha1:WCWJYGVGF7BQYQMKKN5CGUM3UXCLHRK7\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-40/CC-MAIN-2020-40_segments_1600402123173.74_warc_CC-MAIN-20200930075754-20200930105754-00362.warc.gz\"}"}