SHAP Explanations in case of repeated train/test split
I'd propose doing a similar thing that you do with your accuracy and Kappa metrics — calculate SHAP values for all 500 splits, and take the average of these $n\_samples \times n\_features \times 500$ matrixes in the third dimension to get $n\_samples \times n\_features$ matrixes, which you can use to create your desired plots.