high root mean square error in regression model
RMSE doesn't work that way. An RMSE of 13 might actually great, it completely depends on how your target variable is scaled. For example, if your target variable was in the range [0,1e9], than an RMSE of 13 is spectacular. On the other hand, if your target is in the range [0,1], an RMSE of 0.5 is terrible. If you want to try a metric that can be more readily interpretable as having a "good" or "bad" score, try Mean Average Percent Error (MAPE).

As far as why you get a lower MSE when you cross validate: you don't show us how you constructed your training and test sets, but my guess is that you basically just got unlucky and ended up with a training/test split that performed poorly on your holdout set. Your CV-MSE is clearly better than your single holdout MSE, but you should also check the spread of CV scores as well. In any event, for a dataset as small as yours I'd recommend using bootstrap cross validation instead of k-fold.