Kfold or cross_val_score I am all new with ML. I try to understand what is Kfold and cross_val_score.
I made this model:
RandomForestRegressor(max_depth=17,n_estimators=93,criterion='mse')
modelfinal.fit(xtrain1, ytrain1)
mrse_test = np.sqrt(mean_squared_error(y_pred=modelfinal.predict(xtest1), y_true=ytest1))

It gave me:




Train/test
R2
mae
mrse




Train
0.94
13.52
22.07


Test
0.605
34.57
56.592




Because on the train, it gave me 0.9 as R2 and only 0.6 on test.
I have been told  I should use Kfold cv to see if I have over overfitting or not.
But I just can't understand how it will help me, and what I did is right or not.
This is what I did:
cv=ShuffleSplit(5,test_size=0.2)
score=cross_val_score(modelfinal, X, Y, cv=cv)

This is what I get:
array([0.59618812, 0.57799759, 0.55214982, 0.63904499, 0.65027832])

What can I conclude from this?
Overfitting is visible when the performance on the training set is much higher than the test set, so you clearly had overfitting with your first experiment already.
Cross-validation is not a way to see overfitting by itself, it's a way to obtain a more reliable performance value by using several splits (removes chance factor) and using an overall larger test set (the full data).
One thing I notice in your experiment is that the max_depth parameter is set to 17 which is very high, especially in case your training set is not too large. This means that you're allowing the model to create very big and complex trees, so there is more chance that it would represent small patterns which happen by chance in the data. Try reducing this parameter value, hopefully this will avoid or at least decrease the overfitting. If it's not enough there might be some work to do at the level of features: too many features and/or not enough instances can cause overfitting as well.