How to make multiple regression perform better for outliers? (without reducing effect of them) I have a small dataset(about 60 samples) and I need it to predict well for high target values. There are only a few high values and all models I tried perform poorly for these high values. 

So I wonder what technics exist to make algorithm perform better for high values that can't be dismissed as outliers. You see, these few high values make MSE very large, because the models tends to underestimate these high values, predicting them to be 2 or more times smaller.

I have an idea to generate fake data for outliers, but I haven't found how to do it the right way for regression. Is it right to generate fake data for high values based on their proximity by mixing their features? So it would be smth like SMOTE, but instead of classes we have nearby values?

Or, perhaps, the other idea is to cluster targets by density and then to generate balanced clusters by generating fake data with SMOTE or ADASYN?

P.S. Note that I don't want to reduce effect of outliers. On the opposite, these high values are extremely important, so I want model to perform well for them.
When it comes to outliers, one possible way of reducing the effects of the same is by means of a weighting mechanism.

When a standard regression is run on an entire dataset, all the observations in the dataset are being weighted equally.

Therefore, you could choose to run a robust regression where you use a weighting system, e.g. Huber, Bisquare, to adjust the weights so as to place a smaller weight on outliers in your dataset.

One way of detecting outliers in the first instance can be through the use of Cook's Distance, whereby if the distance is greater than 1 (as a rule of thumb), then the observation could be deemed to be an outlier.

You might find the following references helpful:


ROBUST REGRESSION | R DATA ANALYSIS EXAMPLES
Robust Regression — Reduce Outlier Effects