Does the GridSearchCV always improves model accuracy for classification or regression model?
No, it is not necessary that a gridsearchcv will always improve the model performance. It depends upon the hyperparameter range you select for the model and the underlying data.
In some cases, the default parameter itself will be the best, so the best way to further tune is to have the hyperparameters lying around that default range. If either doesn't work, you have to look at the data and preprocessing steps to figure out the problem.
I used GridSearchCV to get the optimized hyperparameter (e.g. learning rate, hidden layer size, activation) for MLPClassifier. but for some datasets, although I applied that parameter to the model, classification accuracy was not improved. for example, the accuracy of datasets1 that include learning rate = a, hidden layer size = b lowered than those of include learning rate = c, hidden layer size = d, optimized hyperparameter obtained from GridSearchCV.
So, I'm confused why this phenomenon happens.