def hyperparameter_validation(X, y, model, hyperparameter, hyperparameter_values, scale=False, test_size=0.2, time_series=False, random_state=123, n_folds=5, regr=True, plot=False, plot_train=False, xvalues=None, xlabel=None, title='Hyperparameter validation', figsize=(6, 6)):
    "\n    Select the best value for the specified hyperparameter of the specified model on the given dataset.\n\n    In other words, perform the tuning of the `hyperparameter` among the values in `hyperparameter_values`.\n\n    This selection is made using the validation score (i.e. the best hyperparameter value is the one with the best validation\n    score).\n    The validation score is computed by splitting the dataset into the training-test sets and then by applying the cross\n    validation on the training set.\n    Additionally, the training and test scores are also computed.\n\n    Optionally, the validation scores of the `hyperparameter_values` can be plotted, making a graphical visualization of the\n    selection.\n\n    Parameters\n    ----------\n    X: np.array\n        Two-dimensional np.array, containing the explanatory features of the dataset.\n    y: np.array\n        Mono dimensional np.array, containing the response feature of the dataset.\n    model: sklearn.base.BaseEstimator\n        Model which has the specified `hyperparameter`.\n    hyperparameter: str\n        The name of the hyperparameter that has to be validated.\n    hyperparameter_values: list\n        List of values for `hyperparameter` that have to be taken into account in the selection.\n    scale: bool\n        Indicates whether to scale or not the features in `X`.\n        (The scaling is performed using the sklearn MinMaxScaler).\n    test_size: float\n        Decimal number between 0 and 1, which indicates the proportion of the test set.\n    time_series: bool\n        Indicates if the given dataset is a time series dataset (i.e. dataset indexed by days).\n        (This affects the computing of the validation score).\n    random_state: int\n        Used in the training-test splitting of the dataset.\n    n_folds: int\n        Indicates how many folds are made in order to compute the k-fold cross validation.\n        (It's used only if `time_series` is False).\n    regr: bool\n        Indicates if it's either a regression or a classification problem.\n    plot: bool\n        Indicates whether to plot or not the validation score values.\n    plot_train: bool\n        Indicates whether to plot also the training scores.\n        (It's considered only if `plot` is True).\n    xvalues: list (in general, iterable)\n        Values that have to be put in the x axis of the plot.\n    xlabel: str\n        Label of the x axis of the plot.\n    title: str\n        Title of the plot.\n    figsize: tuple\n        Two dimensions of the plot.\n\n    Returns\n    ----------\n    train_val_scores: np.array\n        Two dimensional np.array, containing two columns: the first contains the training scores, the second the validation\n        scores.\n        It has as many rows as the number of values in `hyperparameter_values` (i.e. number of values to be tested).\n    best_index: int\n        Index of `hyperparameter_values` that indicates which is the best hyperparameter value.\n    test_score: float\n        Test score associated with the best hyperparameter value.\n    ax: matplotlib.axes.Axes\n        The matplotlib Axes where the plot has been made.\n        If `plot` is False, then it is None.\n\n    Notes\n    ----------\n    - If `regr` is True, the validation scores are errors (MSE, i.e. Mean Squared Errors): this means that the best\n      hyperparameter value is the one associated with the minimum validation score.\n      Otherwise, the validation scores are accuracies: this means that the best hyperparameter value is the one associated\n      with the maximum validation score.\n    - If `time_series` is False, the training-test splitting of the dataset is made randomly. In addition, the cross\n      validation strategy performed is the classic k-fold cross validation: the number of folds is specified by `n_folds`.\n      Otherwise, if `time_series` is True, the training-test sets are simply obtained by splitting the dataset into two\n      contiguous parts. In addition, the cross validation strategy performed is the sklearn TimeSeriesSplit.\n    "
    param_grid = {hyperparameter: hyperparameter_values}
    (params, train_val_scores, best_index, test_score) = hyperparameters_validation(X, y, model, param_grid, scale=scale, test_size=test_size, time_series=time_series, random_state=random_state, n_folds=n_folds, regr=regr)
    ax = None
    if plot:
        if (not xvalues):
            xvalues = hyperparameter_values
        if (not xlabel):
            xlabel = hyperparameter
        ax = _plot_TrainVal_values(xvalues, train_val_scores, plot_train, xlabel, title, figsize)
    return (train_val_scores, best_index, test_score, ax)
Select the best value for the specified hyperparameter of the specified model on the given dataset.

In other words, perform the tuning of the `hyperparameter` among the values in `hyperparameter_values`.

This selection is made using the validation score (i.e. the best hyperparameter value is the one with the best validation
score).
The validation score is computed by splitting the dataset into the training-test sets and then by applying the cross
validation on the training set.
Additionally, the training and test scores are also computed.

Optionally, the validation scores of the `hyperparameter_values` can be plotted, making a graphical visualization of the
selection.

Parameters
----------
X: np.array
    Two-dimensional np.array, containing the explanatory features of the dataset.
y: np.array
    Mono dimensional np.array, containing the response feature of the dataset.
model: sklearn.base.BaseEstimator
    Model which has the specified `hyperparameter`.
hyperparameter: str
    The name of the hyperparameter that has to be validated.
hyperparameter_values: list
    List of values for `hyperparameter` that have to be taken into account in the selection.
scale: bool
    Indicates whether to scale or not the features in `X`.
    (The scaling is performed using the sklearn MinMaxScaler).
test_size: float
    Decimal number between 0 and 1, which indicates the proportion of the test set.
time_series: bool
    Indicates if the given dataset is a time series dataset (i.e. dataset indexed by days).
    (This affects the computing of the validation score).
random_state: int
    Used in the training-test splitting of the dataset.
n_folds: int
    Indicates how many folds are made in order to compute the k-fold cross validation.
    (It's used only if `time_series` is False).
regr: bool
    Indicates if it's either a regression or a classification problem.
plot: bool
    Indicates whether to plot or not the validation score values.
plot_train: bool
    Indicates whether to plot also the training scores.
    (It's considered only if `plot` is True).
xvalues: list (in general, iterable)
    Values that have to be put in the x axis of the plot.
xlabel: str
    Label of the x axis of the plot.
title: str
    Title of the plot.
figsize: tuple
    Two dimensions of the plot.

Returns
----------
train_val_scores: np.array
    Two dimensional np.array, containing two columns: the first contains the training scores, the second the validation
    scores.
    It has as many rows as the number of values in `hyperparameter_values` (i.e. number of values to be tested).
best_index: int
    Index of `hyperparameter_values` that indicates which is the best hyperparameter value.
test_score: float
    Test score associated with the best hyperparameter value.
ax: matplotlib.axes.Axes
    The matplotlib Axes where the plot has been made.
    If `plot` is False, then it is None.

Notes
----------
- If `regr` is True, the validation scores are errors (MSE, i.e. Mean Squared Errors): this means that the best
  hyperparameter value is the one associated with the minimum validation score.
  Otherwise, the validation scores are accuracies: this means that the best hyperparameter value is the one associated
  with the maximum validation score.
- If `time_series` is False, the training-test splitting of the dataset is made randomly. In addition, the cross
  validation strategy performed is the classic k-fold cross validation: the number of folds is specified by `n_folds`.
  Otherwise, if `time_series` is True, the training-test sets are simply obtained by splitting the dataset into two
  contiguous parts. In addition, the cross validation strategy performed is the sklearn TimeSeriesSplit.
def hyperparameter_validation(X, y, model, hyperparameter, hyperparameter_values, scale=False, test_size=0.2, time_series=False, random_state=123, n_folds=5, regr=True, plot=False, plot_train=False, xvalues=None, xlabel=None, title='Hyperparameter validation', figsize=(6, 6)):
    "\n    Select the best value for the specified hyperparameter of the specified model on the given dataset.\n\n    In other words, perform the tuning of the `hyperparameter` among the values in `hyperparameter_values`.\n\n    This selection is made using the validation score (i.e. the best hyperparameter value is the one with the best validation\n    score).\n    The validation score is computed by splitting the dataset into the training-test sets and then by applying the cross\n    validation on the training set.\n    Additionally, the training and test scores are also computed.\n\n    Optionally, the validation scores of the `hyperparameter_values` can be plotted, making a graphical visualization of the\n    selection.\n\n    Parameters\n    ----------\n    X: np.array\n        Two-dimensional np.array, containing the explanatory features of the dataset.\n    y: np.array\n        Mono dimensional np.array, containing the response feature of the dataset.\n    model: sklearn.base.BaseEstimator\n        Model which has the specified `hyperparameter`.\n    hyperparameter: str\n        The name of the hyperparameter that has to be validated.\n    hyperparameter_values: list\n        List of values for `hyperparameter` that have to be taken into account in the selection.\n    scale: bool\n        Indicates whether to scale or not the features in `X`.\n        (The scaling is performed using the sklearn MinMaxScaler).\n    test_size: float\n        Decimal number between 0 and 1, which indicates the proportion of the test set.\n    time_series: bool\n        Indicates if the given dataset is a time series dataset (i.e. dataset indexed by days).\n        (This affects the computing of the validation score).\n    random_state: int\n        Used in the training-test splitting of the dataset.\n    n_folds: int\n        Indicates how many folds are made in order to compute the k-fold cross validation.\n        (It's used only if `time_series` is False).\n    regr: bool\n        Indicates if it's either a regression or a classification problem.\n    plot: bool\n        Indicates whether to plot or not the validation score values.\n    plot_train: bool\n        Indicates whether to plot also the training scores.\n        (It's considered only if `plot` is True).\n    xvalues: list (in general, iterable)\n        Values that have to be put in the x axis of the plot.\n    xlabel: str\n        Label of the x axis of the plot.\n    title: str\n        Title of the plot.\n    figsize: tuple\n        Two dimensions of the plot.\n\n    Returns\n    ----------\n    train_val_scores: np.array\n        Two dimensional np.array, containing two columns: the first contains the training scores, the second the validation\n        scores.\n        It has as many rows as the number of values in `hyperparameter_values` (i.e. number of values to be tested).\n    best_index: int\n        Index of `hyperparameter_values` that indicates which is the best hyperparameter value.\n    test_score: float\n        Test score associated with the best hyperparameter value.\n    ax: matplotlib.axes.Axes\n        The matplotlib Axes where the plot has been made.\n        If `plot` is False, then it is None.\n\n    Notes\n    ----------\n    - If `regr` is True, the validation scores are errors (MSE, i.e. Mean Squared Errors): this means that the best\n      hyperparameter value is the one associated with the minimum validation score.\n      Otherwise, the validation scores are accuracies: this means that the best hyperparameter value is the one associated\n      with the maximum validation score.\n    - If `time_series` is False, the training-test splitting of the dataset is made randomly. In addition, the cross\n      validation strategy performed is the classic k-fold cross validation: the number of folds is specified by `n_folds`.\n      Otherwise, if `time_series` is True, the training-test sets are simply obtained by splitting the dataset into two\n      contiguous parts. In addition, the cross validation strategy performed is the sklearn TimeSeriesSplit.\n    "
    param_grid = {hyperparameter: hyperparameter_values}
    (params, train_val_scores, best_index, test_score) = hyperparameters_validation(X, y, model, param_grid, scale=scale, test_size=test_size, time_series=time_series, random_state=random_state, n_folds=n_folds, regr=regr)
    ax = None
    if plot:
        if (not xvalues):
            xvalues = hyperparameter_values
        if (not xlabel):
            xlabel = hyperparameter
        ax = _plot_TrainVal_values(xvalues, train_val_scores, plot_train, xlabel, title, figsize)
    return (train_val_scores, best_index, test_score, ax)