sklearn's cross_validate does not work with catboost
I found out that adding xtrain.astype('O') works.

Apparently catboost doesn't work with pandas Categorical dtypes yet: https://github.com/catboost/catboost/issues/814