How to justify a predictor in influencing the outcome? I am working on a prediction (binary classification) problem

Currently I get an AUC score of 85-86 and F1-score of 81

Questions

1) The above performance is based on 6 well-known features

2) Let's say I add another feature and I see my AUC and F1 score improve by 1 or 2 points? So is this the only way to know that it really helps/value addition to the model?

3) How do I justify that these features really help in predicting the output?

4) Is there anyway to prove or validate that adding this feature really helps my model and improves the outcome? Is it only using changes in AUC score?

5) For instance, I can add multiple features and my AUC increases by few decimal points, so can I say that they are important or useful or drive predictions? Yes ofcourse, but adding 10 features (where 6 features really impact the outcome and rest 4 increase the auc only by few decimal points). doesn't really overfit. Am I right? Because my prediction score doesn't go beyond 86
Cross Validation

You had a post where we discussed causality, but with ML models assumption is that data represents your problem entirely and has all the information in it. In other words every pattern that you pickup in your train data you can expect it to behave pretty similar in production, hence with this assumption what you want is to "evaluate the entire train" (thats what you can do with CV.) and if it score good on average over all folds you want to add this change (be it a new feature)