GridSearchCV with MLPRegressor with Scikit learn I'm trying to apply automatic fine tuning to a MLPRegressor with Scikit learn. After reading around, I decided to use GridSearchCV to choose the most suitable hyperparameters. Before that, I've applied a MinMaxScaler preprocessing. The dataset is a list of 105 integers (monthly Champagne sales).

The problem is that for some reason the GridSearchCV isn't operating (at least correctly, I think). When I print the parameters used by the model, appear some values out of the range defined in param_list.

Furthermore, i know that the dataset is too small for a MLP, the idea is program the model now and use it later in a larger dataset. Although, the final dataset is not very large, so I'll be very thankful of hear any idea to improve the accuracy of the model in a small datasets!

Thanks!

Code:

from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import TimeSeriesSplit
from sklearn.model_selection import GridSearchCV
from matplotlib import pyplot
from sklearn.neural_network import MLPRegressor
from sklearn.metrics import mean_squared_error
import pandas as pd

dataset = pd.read_csv('champagne.csv', header=None)

scaler = MinMaxScaler()
scaled_dataset = scaler.fit_transform(dataset)

mlpr = MLPRegressor(max_iter=7000)

param_list = {"hidden_layer_sizes": [1,50], "activation": ["identity", "logistic", "tanh", "relu"], "solver": ["lbfgs", "sgd", "adam"], "alpha": [0.00005,0.0005]}
gridCV = GridSearchCV(estimator=mlpr, param_grid=param_list)

splits = TimeSeriesSplit(n_splits=3)

pyplot.figure(1)
index = 1

for train_index, test_index in splits.split(scaled_dataset):

    training_set = scaled_dataset[train_index]
    testing_set = scaled_dataset[test_index]

    train_index_array = train_index.reshape(-1,1)
    test_index_array = test_index.reshape(-1,1)

    gridCV.fit(train_index_array, training_set)
    predicted = gridCV.predict(test_index_array)
    parameters = mlpr.get_params()

    test_mse = mean_squared_error(testing_set, predicted)

    pyplot.subplot(310 + index)
    pyplot.plot(predicted)
    pyplot.plot([None for i in training_set] + [x for x in testing_set])
    index += 1

    train_index.flatten() 
    test_index.flatten()
It would be helpful to get the ouput of the program (or at least the error thrown)

However, MLPRegressor hidden_layer_sizes is a tuple, please change it to:

param_list = {"hidden_layer_sizes": [(1,),(50,)], "activation": ["identity", "logistic", "tanh", "relu"], "solver": ["lbfgs", "sgd", "adam"], "alpha": [0.00005,0.0005]}


https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html