Appropriate objective function and evaluation metric when I DO care about outliers?
These are several things you can try:

Use quartic error, $(y - \hat{y})^4$, instead of quadratic error. This is going to penalize a lot big errors, way more than MSE. The issue is that this is not implemented in xgboost, and you would need to develop a custom loss.
If your target is always positive, you can use the target as training weights. This will give more weights to the outliers. If it is not always positive, you can use the absolute value of the target as weights. If using the target values directly puts too much weight on the outliers, you might want to transform it (e.g. using the log or square root), and if you have samples whose target value is zero, you might want to add some epsilon to all the weights. Note that xgboost can be easily trained using weights.
Try to predict the quantile of the training distribution, then transform your predictions using the training cumulative probability function.