Cross-validation average score
Your result is really a bit strange (it’s the R2, right, so the score makes no sense as R2 should be in a range between 0,1). When you do 10fold cv, each bit of your data will be used in one of the folds. So when 9/10 runs are okay, but one of the 10 scores in a 10th run is very bad, it could be coincidental clustering of your outliers in this one case. 

For me this raises the question of robustness. So I would run „a lot“ of 10fold cv over my data to see if the problem occurs again (and to what extent). This is cheap in your case but gives you good arguments.

Also this gives you a good idea of how sensitive your model actually is to potential „strange“ outliers which might be part of real world problems when I understand you correctly.

If true, I would also check where outliers are particularly problematic if possible. I don‘t know the scale of your model, but having a look at predicted values vs actual y, may give you an idea.