Why are results of cross-validation before splitting is different from results after splitting? All data in my binary classification problem are represented by X and y.  Now, I do stratified cross-validation on this data as follows:
scoring = {'accuracy' : make_scorer(accuracy_score), 
           'precision' : make_scorer(precision_score),
           'recall' : make_scorer(recall_score), 
           'f1_score' : make_scorer(f1_score)}

model=RandomForestClassifier(n_estimators=50,random_state=10) 

results = cross_validate(estimator=model, X=X, y=y, cv=10, scoring=scoring)


If run the code, I will get the following results:
Accuracy : 0.5436815489342804
Precision : 0.020165565854870747
Recall : 0.11013513513513513
F1_score : 0.03315023853741518

Now, I split X and y as follows:
#Test, training data split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20, stratify = y)

#Split the training data into validation set 
X_val_train, X_val_test, y_val_train, y_val_test = train_test_split(X_train, y_train, test_size = 0.1, random_state=0, stratify=y_train )

Now, if I do the same cross-validation procedure like before on X_train and X_train, I will get the following results:
Accuracy : 0.8424393681243558
Precision : 0.47658195862621017
Recall: 0.1964997354963851
F1_score : 0.2773991741912054

I do not understand why results are so different and why this happened.
Without having more information my hypothesis the complexity of your model.
I think you might have a model so complex that it needs a &quot;large&quot; amount of data for having a good performance (First scenario with cross validation on full data)
Then when you split your data in train - test set and repeat the cross validation the amount of data is reduced naturally so it is causing that your model do not have enough samples to generalize.
That might be confirmed via a learning curve:


If the training and cross-validation scores converge together as more data is added (shown in the left figure), then the model will probably not benefit from more data. If the training score is much greater than the validation score then the model probably requires more training examples in order to generalize more effectively.