hyperparameter tuning with validation set
It seems to me that you're manually iterating through the hyper-parameters.

scikit-learn has a number of helper functions that make it easy to iterate through all of the parameters using various strategies:
https://scikit-learn.org/stable/modules/grid_search.html#grid-search

Secondly, if I was 'manually' tuning hyper-parameters I'd split my data into 3: train, test and validation (the names aren't important)

I'd change my hyper-parameters, train the model using the training data, test it using the test data.  I'd repeat this process until I had the 'best' parameters and then finally run it with the validation data as a sanity check (should have similar scores).

With scikit-learn's helper functions, I just split the data into two parts.  Use GridSearchCV with one part and then at the end using the best parameters (stored in the attribute best_estimator_) run a sanity check with the second part.

# define parameter sweep
param_grid = { 'max_iter' : [100, 1000, 10000] }

# define grid search
clf = GridSearchCV(mlp, param_grid, cv=5)

# perform search
clf.fit(X, y)

# best estimator
clf._best_estimator


For info, every estimator and their respective scores are also available as attributes (see https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV) which means you can check the result of every CV if you wanted to make sure you got the best parameters.

As you can see you can avoid thinking too hard about whether CV was the right approach by using a final validation set as a sanity check.